meganz / MEGAsync

Easy automated syncing between your computers and your MEGA Cloud Drive
Other
1.63k stars 280 forks source link

MEGASyncOverlayPlugin spamming system log with entries #524

Open richardstevenhack opened 3 years ago

richardstevenhack commented 3 years ago

I use MEGA primarily for downloads. I don't sync any directories at all and set that in the installer IIRC.

However, when examining my system logs on my openSUSE Leap 15.2 Linux system, I see literally thousands of messages like these: "Dec 21 01:28:00 localhost.localdomain dolphin[2899]: MEGASYNCOVERLAYPLUGIN: getOverlays </home/rhack/.local/share/inxi>: 9"

There appears to be one for every directory on the system (and I have thousands over terabytes of data partitions).

I have set the advanced setting "Disable overlay icons" - but these messages still keep appearing.

I would like to know how to stop them as it bloats my system logs, thus making it much harder to analyze problems on the system using the logs.

richardstevenhack commented 3 years ago

I now suspect that this behavior is causing my system to have a hard freeze at random times. Follow me here:

Over the last couple weeks since reinstalling my openSUSE Leap 15.2 Linux, I've had a number of hard system freezes: the entire system simply freezes solid, no cursor, no continued system behavior. A hard reboot is required. I went on the forums for OpenSUSE and the solution suggested was to get off the Preempt Kernel and on to the Default Kernel, which I did. I then had six days of no freezes.

When I was having freezes, on at least two occasions they occurred when I was doing nothing on the system except using youtube-dl to download some Youtube videos, while lying back listening to music playing on KPlayer. I just had another freeze in exactly the same circumstances. As they said in "Goldfinger", "Once is happenstance, twice is coincidence, three times is enemy action."

From my journal log absolutely nothing is happening on the system on a repetitive basis at the moment of the freeze except the Megasyncoverlayplugin is doing this for each file being downloaded by youtube-dl:

Dec 28 22:10:43 localhost.localdomain dolphin[2913]: MEGASYNCOVERLAYPLUGIN: getOverlays </Data2/Work/Cane Self Defense - How To Defend Against An Attack.f136.mp4.part-Frag403.part>: 9 Dec 28 22:10:42 localhost.localdomain dolphin[2913]: MEGASYNCOVERLAYPLUGIN: getOverlays </Data2/Work/Cane Self Defense - How To Defend Against An Attack.f136.mp4.part-Frag402.part>: 9 Dec 28 22:10:42 localhost.localdomain dolphin[2913]: MEGASYNCOVERLAYPLUGIN: getOverlays </Data2/Work/Cane Self Defense - How To Defend Against An Attack.f136.mp4.part-Frag401.part>: 9 Dec 28 22:10:41 localhost.localdomain dolphin[2913]: MEGASYNCOVERLAYPLUGIN: getOverlays </Data2/Work/Cane Self Defense - How To Defend Against An Attack.f136.mp4.part-Frag400.part>: 9 Dec 28 22:10:40 localhost.localdomain dolphin[2913]: MEGASYNCOVERLAYPLUGIN: getOverlays </Data2/Work/Cane Self Defense - How To Defend Against An Attack.f136.mp4.part-Frag399.part>: 9 Dec 28 22:10:39 localhost.localdomain dolphin[2913]: MEGASYNCOVERLAYPLUGIN: getOverlays </Data2/Work/Cane Self Defense - How To Defend Against An Attack.f136.mp4.part-Frag398.part>: 9 Dec 28 22:10:38 localhost.localdomain dolphin[2913]: MEGASYNCOVERLAYPLUGIN: getOverlays </Data2/Work/Cane Self Defense - How To Defend Against An Attack.f136.mp4.part-Frag397.part>: 9 D

It was at 22:10:44 exactly that the system froze solid.

Now of course this isn't proof of anything. What I'm going to do is disable the megasync service for the next few days except when I need it and see if I get another freeze.

And I'd still like the plugin to be capable of being disabled or at least not reporting its every action to the system logs for every file on my multi-terabyte system.

Someone please respond to this bug report.

ochen1 commented 3 years ago

Try launching file your manager with >/dev/null 2&>1 appended to the end of the command.

eg:

dolphin >/dev/null 2&>1

It silences stdout and stderr, so the messages for logging aren't visible in the terminal/console.

image

richardstevenhack commented 3 years ago

I'm a bit confused. I do have the plugin service active in my dolphin settings. Are you saying that is the source of the messages going to the system logs (not the terminal/console)? I thought they were coming from the MEGASync utility itself while it was running in the system tray. If that's the case, I can probably just disable the Mega service in dolphin and get rid of the messages. I'll try that. I have deactivated the Megasync utility itself and prevented it from automatically loading on boot. I'll deactivate the dolphin service, reactivate the Megasync utility and see if I get any more freezes.

ochen1 commented 3 years ago

Yeah. There is a service called "MEGASync". Just disable that. I'm pretty sure then it'd stop printing to stdout/stderr.

image

richardstevenhack commented 3 years ago

Yes, I did that. What I'm asking is whether it is the Dolphin service that issues messages to the system logs (not stdout/stderr) or is it the Megasynch utility itself that runs in the system tray? I suppose I can find out by running the Megasync utility with the Dolphin service off. I'll try that and check the logs.

richardstevenhack commented 3 years ago

I did run with the dolphin service disabled, with the Megasync utility disabled, and nothing showing in the system tray.

Guess what? I had another freeze. And guess what? journalctl for the exact moment of the freeze showed the megasyncoverlay plugin running! I said to myself, "How the hell is that possible?" On reboot, I examined the running services list and can't see the plugin running.

So I did another Google search and found a reference to where the Megasync services menu is stored in the system. In so doing I was reminded that the Megasync dolphin integration software was installed. Remembering that, I uninstalled the dolphin integration as I never use it - I only installed it on the off chance that I might. So it appears that the dolphin integration software runs by itself independent of the Megasync utility - and it's spamming the logs and causing a system freeze in KDE.

It somehow causes the system freeze by trying to get the "overlay" on files that youtube-dl is downloading, which are fragmentary and temporary files the youtube-dl creates and destroys as part of its download process. This needs to be looked at by the developers! It also spam the system logs with an insane number of messages and that needs to be a prominently displayed user selectable option.

Turning off the stdout/stderr might work, but I'm hardly going to be running Dolphin from the command line every time I want to use it. So uninstalling the dolphin integration would appear to be the way to go until this is fixed.

So now I'm going to run the Megasync utility in the system tray as usual without dolphin integration and see if I get any more freezes. I'll report back the results probably within a week or so. But at this juncture I think I know what is happening.

richardstevenhack commented 3 years ago

Well, I uninstalled the dolphin-integration package, and then re-enabled the Megasync utility which is running in the system tray.

And I got a freeze. And this time journalctl doesn't show anything related to Mega at all that I can see.

I'm going to disable the Megasync utility to prevent startup again and see if I still get a freeze. I'm beginning to suspect youtube-dl and youtube-dl-gui as that's what was I was doing at the time of the freeze. Maybe it's a hardware issue given the totality of the freeze.

However, the Dolphin integration package still spams the logs and that needs to be corrected even if it's not causing my freezes.

I'll report back if I still get a freeze after dis-enabling startup of the Megasync utility which will remove that as a suspect in the freezes.

biggestsonicfan commented 2 years ago

Quite a bump, but my system froze dead. First the graphics cut out, then the audio that was playing soon followed. Decided to check my logs and this is almost exactly the same situation I'm in. Lots of logs from "MEGASYNCOVERLAYPLUGIN" for files and folders completely unrelated to MegaSync, folders MegaSync has never touched, and I assume it's all because of this Dolphin plugin.

While I can't exactly attribute MegaSync to crashing my X server session, it is flooding my logs.

biggestsonicfan commented 2 years ago

System hardlocked again while MegaSYNC was actively scanning a folder that was being moved :-1:

Shin-Aska commented 7 months ago

Workaround for me is I edited the Dolphin.desktop file where I changed the entry from

Exec=dolphin %u

To

Exec=dolphin %u >/dev/null 2>&1

biggestsonicfan commented 7 months ago

Exec=dolphin %u >/dev/null 2>&1

I feel that's an extremely unwise and desperate choice just to mitigate noise in the logs while MEGASyncOverlayPlugin still scans the folders regardless. For starters, you will have to use that desktop shortcut for all instances of Dolphin. If another application launches dolphin, the noise will begin again such as an "Open with File Explorer" option. Also if something catastrophic happens and you need to figure out what happened to undo damage... well you won't be able to.

I highly recommend you undo this and scrub your logs if it bothers you this much and not take this route. Or better yet, uninstall the plugin.

Shin-Aska commented 7 months ago

Exec=dolphin %u >/dev/null 2>&1

I feel that's an extremely unwise and desperate choice just to mitigate noise in the logs while MEGASyncOverlayPlugin still scans the folders regardless. For starters, you will have to use that desktop shortcut for all instances of Dolphin. If another application launches dolphin, the noise will begin again such as an "Open with File Explorer" option. Also if something catastrophic happens and you need to figure out what happened to undo damage... well you won't be able to.

I highly recommend you undo this and scrub your logs if it bothers you this much and not take this route. Or better yet, uninstall the plugin.

My workaround works for me. Especially when the shortcut you need to edit is from .local/share/applications. That supression works when launching Dolphin via a pinned Taskbar or KDE's application launcher. Since I leave Dolphin open almost all the time just as I start my computer (which means that if another application launchers Dolphin, the already opened Dolphin will intercept that instead of opening a new instance) and I don't save sessions.

If I am really desperate to fix that logging problem, I'll just download this source code and just remove this line and recompile.

https://github.com/meganz/MEGAsync/blob/d29d1767e24d85ea64eac6c3de7009b3a2b9536e/src/MEGAShellExtDolphin/megasync-plugin-overlay.cpp#L192

I mean if we think about it, why is Dolphin outputting a debug message when it is not even in debug mode.

But I can't be bothered, the workaround is fast, no need to install crap just to compile a hotfix on my machine

Finally I don't think Megasync scanning one or more of your folders is the reason why your X crashes. I have been using this plugin for months now and never experience a crash or system lock. Could be a Dolphin bug or somewhere else.

Edit:

I also want to point out that uninstalling the plugin is a no deal for me. I want an indicator when my files are synced and I have been using this for months. I just don't like my syslog growing to ridiculous sizes just because of a spammed log and I certainly do not want to always be cleaning the logs all the time.

biggestsonicfan commented 7 months ago

This issue was greater than noise in the logs, it's the plugin actively accessing data it has no business accessing and hardfreezing computer systems. Removing debug messaging, as seen here https://github.com/meganz/MEGAsync/issues/524#issuecomment-752896098, still doesn't fix the issue at hand.

EDIT: Since I left linux, I really don't have much more to offer in the thread other than I removed the dolphin plugin and locked the package from reinstalling so I will also unsubscribe from this thread.

Regardless, I think outputting logs to null is reckless.