meganz / MEGAsync

Easy automated syncing between your computers and your MEGA Cloud Drive
Other
1.58k stars 279 forks source link

Does not check for pre-existing files anymore, redownloads existing files wasting download. #789

Open Reinbowsaur opened 1 year ago

Reinbowsaur commented 1 year ago

I believe this was introduced with a recent patch. Files are being re-downloaded without checking for a pre-existing file first, and being appended with a number.

I have verified this many times, using multiple directories and re-installs/reboots of my machine. This is wasting alot of peoples download quotas without them knowing and needs to be addressed asap. (Windows user here in-case its unique to windows client)

Goldmaster commented 1 year ago

I have this issue as well

TheArgentinian commented 1 year ago

Hey devs, are you ever going to fix this tiny issue? I wasted gigabytes on files that won't resume after a PC reboot! How do you fuck up something so basic?!

PowerWasher9000 commented 1 year ago

Yeah, restarting megasync or your computer will reset any unfinished downloads you may have. Kinda annoying.

leo1115 commented 1 year ago

I cant believe this is not being fixed.

c-martin commented 1 year ago

This is a wild bug to leave unaddressed for months. Does anyone know what version it was introduced in? At this point I'm looking at reverting to a previous release and refusing updates...

leo1115 commented 1 year ago

This is a wild bug to leave unaddressed for months. Does anyone know what version it was introduced in? At this point I'm looking at reverting to a previous release and refusing updates...

likely v4.8.9.0 where can you find safe installer file for an older release ?

c-martin commented 1 year ago

where can you find safe installer file for an older release ?

It's not official so I don't know what your threshold for "safe" is, but I did some experimenting with the archived Windows installers here. Downgrading to 4.9.0, 4.8.8, or 4.8.5 did NOT work, but 4.8.1 will properly resume partial downloads. Be warned that your transfer list might get lost, during one of those downgrades I had to re-import the links to the folders I was trying to download.

Goldmaster commented 1 year ago

but 4.8.1 will properly resume partial downloads.

This is the direct link https://megasync.en.uptodown.com/windows/download/86917775

Virustotal link https://www.virustotal.com/gui/file/3c5db4f98ce373725edf4b8ffb83fdedc7341f08c8514aa339e4ce44f4953e80

Just remember to untick the auto update option

okurka commented 1 year ago

It's still not fixed in v4.9.3 that was released today.

Reinbowsaur commented 1 year ago

Friendly reminder that you have lost a potential customer mega. It was already hard for me to consider paying in the first place, and now there is no saving grace to make mega even somewhat usable.

Not only is mega absolute crap in general in the first place with the lack of features and syncing settings, it's just utterly useless now. You sealed the deal on how crap mega already was if this shit was actually intended.

mattw-mega commented 1 year ago

Hi @okurka , could you please explain your use case, why are you downloading the same file to the same location again? And, is it enough that there is already a file present with the right name or would you want to ensure that the file at that location really is identical to the one in the cloud, or perhaps overwrite it to be sure? thanks

c-martin commented 1 year ago

Hi @okurka , could you please explain your use case, why are you downloading the same file to the same location again? And, is it enough that there is already a file present with the right name or would you want to ensure that the file at that location really is identical to the one in the cloud, or perhaps overwrite it to be sure? thanks

The problem is that resuming interrupted downloads from a hidden temporary file (.getxfer.XXX.mega) simply does not work. If you attempt to download a file and it gets interrupted for any reason, like your computer restarts or megasync restarts or your internet connection changes, megasync will NOT recognize the previously in-progress download the next time it launches, it will attempt to restart downloading that file from 0.

This is especially problematic with single files that are larger than the free transfer limit, since if you need to close the mega app before that 8 hour timeout (or however long it is) expires you will never download the complete file.

mattw-mega commented 1 year ago

Hi @okurka , that bug was fixed, and the fix is present in 4.9.3. I've just double checked in 4.9.3, exiting the app in the middle of a 1gb download, and it resumed fine. The fix did require adding one more field to the transfer database record, if that is not present then it can't resume. So if your 4.9.3 was resuming a download from a prior version, that may be why you saw a case that didn't work. But if it was a 4.9.3 download, 4.9.3 can resume it. Or if there is something more complicated and unusual about your case then please let us know what that is so we can reproduce the issue. thanks

Goldmaster commented 1 year ago

I'm going to stick with 4.8.1.0 until more people confirm there is a fix for this bug. I would only suggest updating during a free day.

leo1115 commented 1 year ago

Just checked the new update (4.9.3) & the downloads now resume as expected.

leo1115 commented 1 year ago

Update: the queue order is not retained if you change the order of files in the download queue & exit the app. @mattw-mega

TheArgentinian commented 1 year ago

Does 4.9.3 work or not? The files that were on the list when you close the app should be there when you open it. Same download progress. Not extra touching should be required from the user end. We're not asking for a fucking miracle here.

1GB is nothing. Try to download a 20gb file and see it restart from the beginning after downloading half of it, 3GB per day and see if you don't get pissed off about it.

TheArgentinian commented 1 year ago

So I installed 4.9.3 expecting to resume what I downloaded. I added the same mega link and saved it to the same location folder but it always starts from zero. Changing the old filename to the new one does nothing.

Sin título

okurka commented 1 year ago

Hi @okurka , that bug was fixed, and the fix is present in 4.9.3. I've just double checked in 4.9.3, exiting the app in the middle of a 1gb download, and it resumed fine. The fix did require adding one more field to the transfer database record, if that is not present then it can't resume. So if your 4.9.3 was resuming a download from a prior version, that may be why you saw a case that didn't work. But if it was a 4.9.3 download, 4.9.3 can resume it. Or if there is something more complicated and unusual about your case then please let us know what that is so we can reproduce the issue. thanks

@mattw-mega This issue isn't about resuming downloads. It's about MEGAsync dowloading files again that were already downloaded.

The use case is downloading a shared folder that gets updated every couple days or weeks. MEGAsync 4.8.5 checks for the files you already downloaded and skips those unless they were updated in the shared folder. MEGAsync 4.9.3 doesn't check for the files and downloads all of them again.

mattw-mega commented 1 year ago

Hi @okurka , thanks for explaining your use case. Although you are downloading to the same folder, MEGAsync doesn't know that so it has to try to decide for each file that would be a collision, is it actually the same file data or not. Some previous code made an approximation at that, based on lots of tiny samples of the file (our FileFingerprint code) (plus mtime/size) but it was not always accurate. So now we have perhaps overcorrected for this case, but you can be sure you really are getting the right file data. We will look into it, I think probably we will add options for you to specify an appropriate method for deciding the files are the same and can be skipped. thanks

okurka commented 1 year ago

@mattw-mega I don't think it checks anything in 4.9.3.

I setup a test folder with several small FF-filled files that I won't change at https://mega.nz/folder/bNkA3CrZ#1x5qLYhi9sKLP4qs5Up6Qg

4.8.5 skips the files that were already downloaded, as expected, but 4.9.3 downloads all those files again and saves them as a copy.

mattw-mega commented 1 year ago

Hi @okurka , thanks for the demo folder, and I see what you're saying. However, although you know you haven't changed the files there, on your second download MEGAsync hasn't recorded that or any info about what changed locally or remotely. It's just starting a folder download to a non-empty target and has to figure it out as it goes based on what it can see in the cloud and local folders. I've prepared a shared folder that demonstrates the issue in 4.8.1, to clarify this case and hopefully justify this change and set your mind and others at ease. This link contains 'before' and 'after' folders, each with a folder to download (simulating your case of a second download to the same location). They both contain copies of your 65536 byte file. However, in the 'after' folder, most of the files have a single byte changed at offset 0xA0, and with the same mtime as the originals. This sort of change is of course very rare overall but can occur naturally if you're working with code and using a tool like git that resets mtimes. If you try to download the 'after' folder's subfolder over the top (using 4.8.1), it will think the files are unchanged, and declare the operation complete without actually downloading any of them. https://mega.nz/folder/ZBsBDLDY#icL5b1CpEWYOc18TDvAlhg/folder/cAlCVYYY. So I think for your case, the option that we will add that will solve it is that we'll do a cryptographic comparison of the entire local file - reading the entire file off disk, calculating the cryptographic MAC and comparing that to the MAC recorded in the corresponding cloud Node. It's more disk and CPU but probably required for your case, and you will be able to opt in to that. thanks

okurka commented 1 year ago

FYI, 4.9.4 still doesn't have this option, so keep using 4.8.5

okurka commented 1 year ago

FYI, it still isn't fixed in 4.9.5.

TheArgentinian commented 11 months ago

Just lost 19GB because this piece of shit software removes the file from the dl list when you run out of disk space and if you re-add the link, it doesn't resume.

There has to be an alternative to Megasync

ILogOutOnTheToilet commented 11 months ago

I also have the same issue. Had to go back into a Mega link and manually compare and select files I haven't downloaded yet.

FYI, 4.9.4 still doesn't have this option, so keep using 4.8.5

@okurka How do you download the official Mega app version 4.8.5?

okurka commented 11 months ago

There's a link to older releases at the top of this thread. https://github.com/meganz/MEGAsync/issues/789#issuecomment-1516728749

The problem seems to be fixed in 4.9.6.

BoiGeezums commented 10 months ago

@mattw-mega So is there a way to force Mega to recognise a partial download between versions? Or have I just wasted hours and 9GB of data? Update took effect just now after restarting my PC and it's not recognising the partially downloaded file. I tried pasting the same link into Mega and it just restarted the download as a new file, similarly to how @TheArgentinian experienced it.

mattw-mega commented 10 months ago

Hi @BoiGeezums , that should be fixed now, please make sure you're running the latest version (4.9.6). If you tried resuming a file that was downloading from an older version (depends which one), then it may not have been able to resume due to some missing data in the database record of that older version. But if you're running the latest, then downloads it is running should be resumable by it, and by future versions. thanks

ILogOutOnTheToilet commented 6 months ago

It still downloads duplicate files by appending with "(1)" for version 4.11.0.

biggestsonicfan commented 3 months ago

Yeah, restarting megasync or your computer will reset any unfinished downloads you may have. Kinda annoying.

It's wild that I had to assume when file were added to a queue, it would persist. This issue doesn't seem to be fixed.

flaccidbagel commented 2 months ago

Posting here to state that this is is something I missed which also caused me to rollback to an earlier version. Any form of shared folder that gets updated constantly is now a massive chore to manage.

Doofy420 commented 1 month ago

For just downloading, jdownloader handles it the right way, it checks the filenames and skips the ones that already exist, obviously it wouldn't know if a file is replaced under the same filename. It's still better than the braindead default behaviour of this app which is to download duplicates of every file and waste your quota. Like stated above, it's for downloading folders that are shared and periodically updated, but their support can't seem to grasp the concept behind this simple thing for some reason, like it's rocket science or something.