meganz / MEGAcmd

Command Line Interactive and Scriptable Application to access MEGA
Other
1.94k stars 407 forks source link

Sync stuck in None state Ubuntu 22.04 #858

Open rserentill opened 1 year ago

rserentill commented 1 year ago

Hi,

I've been using mega-cmd for quite a while now, overall, the sync process works fine, but since 2 days ago the only folder I have synching (a really big one) is not synching anymore. The state of mega-sync shows NONE.

I tried killing mega-cmd-servera couple of times, loging out and loging in again, but nothing seems to work.

htop shows that mega-cmd-server is taking the 100% of 2 out of 4 cores (vCPU).

Can I please get any help on how to solve the issue?

polmr commented 1 year ago

Hi @rserentill . How big is really big and how long did you waited? It seems the sync engine is doing some computation, although that shouldn't last forever.

Perhaps the logs indicate if there is some underlying issue: Instructions to obtain them: Please, close all megacmd instances and execute the server in a separate console: Open a console and type

killall -2 mega-cmd-server
mega-cmd-server --debug-full | tee megacmd.log

Then, open another console and then try to reproduce the issue. Finally quit the server (Ctrl+C in the console that run it should do). Then please send us that megacmd.log file. The log might contain some sensitive data. Feel free to remove anything you don't want to share with us. If you do not want to post it here (you'd rather not), send it to support@mega.nz and refer this issue in your email so that it can reach development team.

Further info regarding Debugging https://github.com/meganz/MEGAcmd/blob/master/contrib/docs/DEBUG.md

rserentill commented 1 year ago

Hey @polmr, thanks for your answer! The folder is around 12GB and it contains ~10k directories and ~70k files. After 4-5 hours, the state is not NONE anymore, it is Syncing, and the SIZE, FILES, DIRS columns update bit by bit, but is very slow (in two days it went from 11.28GB to 11.77GB). The main problem is that these values update when I run mega-sync but I don't see any file being downloaded, also transfers --show-syncs doesn't show anything.

I think I will wait a bit more to see if it starts downloading files before killing all the instances, as you requested, because I don't want to lose any potential progress it made (does that make sense?).

Is this normal? Should it start downloading files at some point?

polmr commented 1 year ago

That is indeed slow.

I don't have any actual theory ...

The increase in size yet no progress may be related to the download of tiny files.

mega-transfers --show-syncs --show-completed

or unlimited listing:

mega-transfers --show-syncs --show-completed --limit=9999999

should display those.

polmr commented 1 year ago

Note that you can increase logging level live without closing MEGAcmd Server:

mega-log DEBUG

and obtain the log file from ~/.megaCmd/megacmdserver.log

rserentill commented 1 year ago

I just ran mega-transfers --show-syncs --show-completed --limit=9999999 and it shows nothing.

Just now the state went back to NONE 😞. I've set the log level to debug, and on themegacmdserver.log file I see a bunch of these:

[API:debug: 08:41:27] Unsynced remote node in syncdown: /path/to/folder Nsize: -1 Nmtime: 0 Nhandle: lUR2mZpC
[API:debug: 08:41:27] doesn't have a previous localnode
[API:debug: 08:41:27] Creating local folder
[API:debug: 08:41:27] Skipping folder creation over an unscanned file/folder, or the file/folder is not to be synced (special attributes)
rserentill commented 1 year ago

I don't see mega-cmd-server processess running in htop anymore, and the state is NONE again. Does this mean it won't sync?

rserentill commented 1 year ago

@polmr I just removed the cache with rm ~/.megaCmd/megaclient_statecache* and now the size is back to 11.30GB and two mega-cmd-server are appearing againt with 100% of CPU usage each. I can't wait 2-3 days more for it to sync less than 1 GB (and not really download any file). I really need this to start working again, I'm starting to be a bit desperate

rserentill commented 1 year ago

@polmr I sent the log file to support@mega.nz last friday.

After two and a half more days it reached 11.77 GB again, which I think is the total size of the folder, and the SyncState is NONE again, but no files were downloaded.

It looks like it is skipping files/folders for some reason, but no files or folders are excluded. This is the output of mega-exclude:

List of excluded names:
.*
Thumbs.db
desktop.ini
~*

I'm trying to access the Cloud Drive via web to see if everything looks good there, but I'm unable to do so because the web page gets stuck while trying to log in (see image below).

Captura de pantalla 2023-07-30 a les 12 39 44

This issue accessing MEGA on the web has been happening for some years now, I've written to support a bunch of times but they never provided a solution. I never insisted much because the clients on the computers (Windows and the one on Ubuntu server) seemed to be working fine and we were able to continue working, until now.

rserentill commented 1 year ago

I finally managed to connect via web and everything looks fine there, all the files are there. It looks like the client on Ubuntu is not downloading anyhting. I also downloaded the iOS app and I think the files are not being downloaded there as well, but they are on the web. Why are the downloads of these files being skipped?