Caching system is unclear/possibly buggy

Overview

[ Desktop(non-portable) ] [ Mounted Drive ]
[ Windows 10 ]

Issue

So let me start off by saying that this isn’t a new ‘issue’, but rather something to look into for how the file caching logic works.
As somebody who doesn’t care for sync folders, I’m only here for the mounted drive and this is the only annoying thing with Icedrive.

The issue with the drive is that caching is weird at best and unclear/buggy at the very worst.

Regardless of Checking/Unchecking “Minimize local disk utilization” big file uploads still do some sort of caching, which makes sense. But the issue is that this cache often isn’t cleared up until restart.

Without checking the “Minimize local disk utilization” and specifying a bigger cache, e.g 100GB, it still goes over this limit if transfering anything that’s over 100GB for example, because it doesn’t do file cycling or backchecking it seems. So why even specify a limit at this point? Seems redundant.
To me this doesn’t look like a cache system, but an unnecessary stop between start-endpoint, that eats up disk space.

There’s also an interaction difference to look into, which is pasting and cutting files.
If a file in transfer encounters an issue due to cache or running out of disk space and it was ‘cut’ instead of ‘pasted’ then it’s effectively lost since it’s probably in some half-encrypted/part style format and unreadable to me.

Please clarify the cache options and please implement cycling into it, so it actually removes the parts/files from cache that have been uploaded.
Preferably it should actually take into account how much disk space there is, right now moving hundred GBs around simply chokes the whole system because it doubles the amount I want to transfer, this can’t be correct.

Did this just now to reproduce, uploaded about 150GB worth of files, Windows install disk was showing ~70GB as free, I used the ‘Clear Cache’ button and it went up to 130GB, but cache folder was still clearly full of temp files, ~140GB free after manually clearing cache files.
Closed the app and opened again and re-mounted, ~220GB free.
This doesn’t make sense.

E: Just in case, yes I’ve tried re-install and even using other disks as cache, it still fills up /tmp/ sometimes and never clears parts/mid-way files.

2 Likes

Hey @neffins thank you for your feedback regarding our cache system for the drive mounted app. I’ll make sure our dev team take a look at this for future updates to the app. Regarding the cache options, could you submit a support ticket via Login - Icedrive and I’ll forward onto our dev team for them to get in touch with you?

I just encountered the exact scenario where I had to terminate a cut to a mounted drive, because it was totally filling up my hard drive. This ruined the files in question, which I then had to re-extract from the encrypted backup file I had extracted them from. No data loss in this case, fortunately, as I had another copy.

That goes to say I second the request for better cache handling.

i have the same scenario V2.70 Windows

@Chris Hey Chris!

Do you guys perhaps have any updates for this? I’ve unfortunately moved from Icedrive as my primary due to this and I’m keeping the service as a backup in hopes that this gets better, because it’s really the only issue.
You guys have been scarce with updates in general, with only one(?) this year.

Not trying to twist any arms here, but other people are clearly trying to use the service in the same way as I am and encountering the same kind of problems so I don’t think it’s user error.

About the logs, I think this should be perfectly reproducible by the devs themselves, just generate 50-100GB worth of random write data with varying sizes of 100MB - 3 GB per file and do the aforementioned moves described in main post and replies. I have a hard time believing it’s specific to my system.

Also, what do your logs contain? I’m using a zero-knowledge cloud for a specific reason, sometimes even my file names can contain a specific ID that are used for internal indexing, therefore I’d rather not even show those. I would have to go through the logs and obfuscate filenames.

This issue is something I’d consider pretty serious for a file host service so I do think it warrants a bit of an acknowledgement that you guys are actually working on it. Actual files can be lost entirely and the cache/limits are not working as intended.

Hey @neffins our dev team are working on redeveloping an entirely new mounted drive app to address a number of concerns and issues that you and others have identified. It’s still in development so can’t give an ETA just yet but it is coming.

Hey @Chris!

That’s good to know, thanks for the update.