meganz / MEGAcmd

Command Line Interactive and Scriptable Application to access MEGA
Other
1.94k stars 407 forks source link

mega-cmd deleting files by itself due to Invalid key/Decryption error #511

Open haelix888 opened 3 years ago

haelix888 commented 3 years ago

I have megacmd 1.4.0-3 on 2 arch linuxes and one of them uploads some files, the other downloads them and randomly deletes some of them while saying this:

[API:warn: 12:15:09] Event 99431: MAC verification failed
[API:debug: 12:15:09] Transfer failed with error -14
[API:warn: 12:15:09] Event 99433: Undecryptable file
[API:debug: 12:15:09] Removing transfer
[API:warn: 12:15:09] Transfer (DOWNLOAD) finished with error: Invalid key/Decryption error File: XXXXX.XXX
[API:debug: 12:15:09] Removing cached transfer

This is obviously a huge issue. The file is deleted (moved to SyncDebris on the sending machine and in the cloud). I have verified the cloud version to be identical to the uploaded file so file correctly uploads.

I have had 25 such files deleted by a receiving synced machine, out of hundreds, over 2 days. How is it possible for the image to be "undecryptable"? and furthermore, how can it be that the receiver takes the decision to delete the offending file?

haelix888 commented 3 years ago

Coming back with more information. There is definitely an issue here.

The problematic files (which were uploaded, but the 2nd machine could not download) are found in Trash > SyncDebris (I see them using the webclient). Attempting to download them from there fails for a subset of them, with the same "Decryption error", see below. I tried logging out and back into the webclient, to refresh the session, but still same behavior.

erorr-with-chrome-and-extension

GitStorageOne commented 3 years ago

Every time when I try to use Mega, something happens with it)
I believe that reported issue exists, similar problems was with sync app, many years ago. Sure it still there.
If you try to use mega "edge" extension, you may find, that it leaves files in browser cache (entire file).
Well, maybe it's better to encrypt files locally with 7z and store it somewhere else with cheapest plans.
By the way, 7z includes command line tool, even for linux. rsync + 7z = true power without side effects like file losses.

haelix888 commented 3 years ago

It must be just a bug, which needs fixing. I'm happy to assist with it. 7z and AWS is no replacement for a sync solution. I'm quite happy with the Mega product otherwise.

haelix888 commented 3 years ago

Again

[API:debug: 23:00:05] Contiguous progress: 5425190 (0) [API:warn: 23:00:05] Event 99431: MAC verification failed [API:debug: 23:00:05] Transfer failed with error -14 [API:warn: 23:00:05] Event 99433: Undecryptable file [API:debug: 23:00:05] Removing transfer [API:warn: 23:00:05] Transfer (DOWNLOAD) finished with error: Invalid key/Decryption error File: asi se canta.mp3

What are we doing with this guys? I can imagine few bugs more critical than arbitrarily dropping files??! A regular user is unlikely to even notice.

xWTF commented 2 years ago

TL;DR: Solution at the end.

Issue

Facing exactly the same issue. Three of my files are disapperaring FOR NO REASON minutes after upload. After fighting the mega desktop for about 2 hours, keep uploading and seeing them disappear, I finally noticed these files are deleted by one of my mega-cmd client with following error message:

[API:warn: 02:03:51] Event 99431: MAC verification failed
[API:debug: 02:03:51] Transfer failed with error -14
[API:warn: 02:03:51] Event 99433: Undecryptable file

After hours of debugging, reinstall, wipe cache, etc., I decided there's something wrong with my data on MEGA's backend server, since the files seems not actually being "uploaded" when I copy them from local backup into the sync folder, instead they're "moved back" from MEGA servers' "cloud trashbin", instead of being re-encrypted and re-uploaded from local. The design totally makes sense, the problem is - when the file located in "cloud trashbin" are corrupted and unable to be decrypted, the loop of "upload"-deleted continues and confuses the user by a lot.

Also tried to upload those files via webui, but the issue persists, I guess it's caused by MEGADesktop app running in background, webui upload was redirected and same logic applies.

Possible reproducion steps

Not exactly sure what steps are required to reproduce this one (and issues appears randomlly sucks, i know). Here's what I'm doing RIGHT BEFORE the issue appears:

I have a 60G folder with 13k files with simple nested structure:

THE_SYNC\
  folder1\
    file1
    file2
    ...
  folder2\
    file3
    file4
    ...
  ...

The folder is synced between a mega-cmd client and a MEGADesktop client, and there're some files with extremely long filename (exceeds btrfs 255byte limit). mega-cmd keeps downloading them and failing, causing a lot of traffic. Thus, I executed a script to rename all of files. The renaming process took about 3s and MEGADesktop took about 10s to sync the changes to cloud. mega-cmd synced those renaming changes to local slowly (I believe it took more than 30min), and I found this issue when I check the file integrity between these two clients.

BTW, here's the environment: MEGAcmd version: 1.4.0.0: code 1040000 - on SynologyNAS DSM 7.0.1-42218 Update 3 MEGAsync 4.6.5 64bit (995bc5c9) - on Windows Server 2022 DataCenter

Solution (for users facing the same issue)

Go to mega's webui, empty the rubbish bin, and re-upload affected files via sync folder from any client (simply drag them into sync folder) And problem solved, mega-cmd is happy to download and decrypt that node now, no more mysterious file disapperaring :)

Hope some mega developer take a look at this @polmr @aabellagm @mattw-mega , thanks.

haelix888 commented 1 year ago

gentle bump