[Bug]: Desktop client fills all available memory and crashes after Google Earth is launched

gessel commented 1 year ago

⚠️ Before submitting, please verify the following: ⚠️

[X] This is a bug, not a question or a configuration issue.
[X] This issue is not already reported on Github (I've searched it).
[X] Nextcloud Server and Desktop Client are up to date. See Server Maintenance and Release Schedule and Desktop Releases for supported versions.
[X] I agree to follow Nextcloud's Code of Conduct

Bug description

The share dialog is looking incomplete. ... This is a duplicate report of #4646 - as the client and server are different and those fields are a big part of the reports, I'm creating this one for your documentation pleasure.

If Nextcloud Desktop Client is running and Google earth is launched (the google earth directory is partially synced, see details) then although the desktop client does not show that the folder is actively being synced, memory usage runs away and the client crashes or the system pushes swap so hard it becomes unusable.

Steps to reproduce

Right click on the file
Click on Share Options
The share dialog does not display the option to share by e-mail ...
Start Nextcloud Desktop Client
Let the client finish syncing and stabilizes with minimal memory footprint (~0.8% of 32GB) (this is stable under most of my use, just not with Google Earth.
Launch Google Earth Desktop
Navigate around, this causes tiles to load - I suspect this is part of the issue. Note that they fill the cache dir, which is unselected for sync and also explicitly ignored memory usage over time 1 second updates

I suspect the cache folder upsets the client's filter routine which leaks memory. The folder might hold 315MB or so of data.

They're explicitly ignored:

and unselected.

Expected behavior

When clicking on the share dialog, share by e-mail should be an option. ... No crash, no crazy memory usage.

Which files are affected by this bug

none - files aren't actually synced, it is a desktop client problem

Operating system

Linux

Which version of the operating system you are running.

jammy Ubuntu 22.04.1 LTS

Package

Other

Nextcloud Server version

24.0.5

Nextcloud Desktop Client version

3.6.2

Is this bug present after an update or on a fresh install?

Updated to a major version (ex. 3.3.6 to 3.4.0)

Are you using the Nextcloud Server Encryption module?

Encryption is Disabled

Are you using an external user-backend?

[ ] Default internal user-backend
[ ] LDAP/ Active Directory
[ ] SSO - SAML
[ ] Other

Nextcloud Server logs

{"reqId":"D0QO4V22AhVAI7oQiit9","level":2,"time":"2022-11-21T10:40:43+00:00","remoteAddr":"re.dac.te.ed","user":"--","app":"core","method":"POST","url":"/","message":"Trusted domain error. \"re.dac.te.ed\" tried to access using \"re.dac.te.ed\" as host.","userAgent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.129 Safari/537.36","version":"24.0.5.1","data":{"app":"core"}}
{"reqId":"YNAPV2ElWnziG60Lqbfb","level":2,"time":"2022-11-21T10:52:16+00:00","remoteAddr":"re.dac.te.ed","user":"--","app":"core","method":"GET","url":"/remote/fgt_lang?lang=/../../../..//////////dev/cmdb/sslvpn_websession","message":"Trusted domain error. \"re.dac.te.ed\" tried to access using \"re.dac.te.ed:443\" as host.","userAgent":"Python-urllib/3.8","version":"24.0.5.1","data":{"app":"core"}}
{"reqId":"XWC0Jhbm7mfkFfiSZo5V","level":2,"time":"2022-11-21T11:15:10+00:00","remoteAddr":"re.dac.te.ed","user":"--","app":"core","method":"HEAD","url":"//","message":"Trusted domain error. \"re.dac.te.ed\" tried to access using \"re.dac.te.ed\" as host.","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36","version":"24.0.5.1","data":{"app":"core"}}
{"reqId":"wNDdbZcWehLT6HFLZ1W4","level":2,"time":"2022-11-21T13:22:33+00:00","remoteAddr":"re.dac.te.ed","user":"--","app":"core","method":"GET","url":"/","message":"Trusted domain error. \"re.dac.te.ed\" tried to access using \"re.dac.te.ed:443\" as host.","userAgent":"--","version":"24.0.5.1","data":{"app":"core"}}
{"reqId":"U1KRWON5Kwpo7amSbmE5","level":2,"time":"2022-11-21T13:22:34+00:00","remoteAddr":"re.dac.te.ed","user":"--","app":"core","method":"GET","url":"/","message":"Trusted domain error. \"re.dac.te.ed\" tried to access using \"re.dac.te.ed\" as host.","userAgent":"Mozilla/5.0 (compatible; CensysInspect/1.1; +https://about.censys.io/)","version":"24.0.5.1","data":{"app":"core"}}
{"reqId":"MUoC0iogtnjCPDdAuhVf","level":2,"time":"2022-11-21T13:22:35+00:00","remoteAddr":"re.dac.te.ed","user":"--","app":"core","method":"GET","url":"//","message":"Trusted domain error. \"re.dac.te.ed\" tried to access using \"re.dac.te.ed\" as host.","userAgent":"Mozilla/5.0 (compatible; CensysInspect/1.1; +https://about.censys.io/)","version":"24.0.5.1","data":{"app":"core"}}
{"reqId":"Hau51UP6h0L94QzyqmFp","level":2,"time":"2022-11-21T14:40:40+00:00","remoteAddr":"re.dac.te.ed","user":"--","app":"core","method":"OPTIONS","url":"/","message":"Trusted domain error. \"re.dac.te.ed\" tried to access using \"re.dac.te.ed\" as host.","userAgent":"--","version":"24.0.5.1","data":{"app":"core"}}
{"reqId":"vN222i9VIUPxEXIEwyVU","level":2,"time":"2022-11-21T16:59:47+00:00","remoteAddr":"re.dac.te.ed","user":"--","app":"core","method":"GET","url":"//","message":"Trusted domain error. \"re.dac.te.ed\" tried to access using \"re.dac.te.ed\" as host.","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:47.0) Gecko/20100101 Firefox/47.0","version":"24.0.5.1","data":{"app":"core"}}
{"reqId":"lr46LfCPj5d1hUa4wT78","level":2,"time":"2022-11-21T17:52:14+00:00","remoteAddr":"re.dac.te.ed","user":"re.dac.te.ed","app":"user_ldap","method":"POST","url":"/index.php/apps/user_ldap/ajax/getConfiguration.php","message":"Configuration Error (prefix s01): No LDAP Host given!","userAgent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36","version":"24.0.5.1","data":{"app":"user_ldap"}}
{"reqId":"lr46LfCPj5d1hUa4wT78","level":2,"time":"2022-11-21T17:52:14+00:00","remoteAddr":"re.dac.te.ed","user":"re.dac.te.ed","app":"user_ldap","method":"POST","url":"/index.php/apps/user_ldap/ajax/getConfiguration.php","message":"Configuration Error (prefix s01): No LDAP Port given!","userAgent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36","version":"24.0.5.1","data":{"app":"user_ldap"}}
{"reqId":"lr46LfCPj5d1hUa4wT78","level":2,"time":"2022-11-21T17:52:14+00:00","remoteAddr":"re.dac.te.ed","user":"re.dac.te.ed","app":"user_ldap","method":"POST","url":"/index.php/apps/user_ldap/ajax/getConfiguration.php","message":"Configuration Error (prefix s01): No LDAP Login Filter given!","userAgent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36","version":"24.0.5.1","data":{"app":"user_ldap"}}
{"reqId":"lr46LfCPj5d1hUa4wT78","level":2,"time":"2022-11-21T17:52:14+00:00","remoteAddr":"re.dac.te.ed","user":"re.dac.te.ed","app":"user_ldap","method":"POST","url":"/index.php/apps/user_ldap/ajax/getConfiguration.php","message":"Configuration Error (prefix s01): Not a single Base DN given.","userAgent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36","version":"24.0.5.1","data":{"app":"user_ldap"}}
{"reqId":"lr46LfCPj5d1hUa4wT78","level":2,"time":"2022-11-21T17:52:14+00:00","remoteAddr":"re.dac.te.ed","user":"re.dac.te.ed","app":"user_ldap","method":"POST","url":"/index.php/apps/user_ldap/ajax/getConfiguration.php","message":"Configuration Error (prefix s01): login filter does not contain %uid place holder.","userAgent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36","version":"24.0.5.1","data":{"app":"user_ldap"}}

Additional info

It would be very helpful to have an automatic redaction tool for log posts - usernames, IP addresses, filenames, directory paths. Hashing them could be sufficient, even unsalted, as rainbow table precomputes are unlikely to be worth the CPU, but it an awful lot of extremely sensitive data is bundled into the logs/debug archives.

jlynnes commented 1 year ago

I have the same issue, but not including Google Earth. On two separate windows clients (win11 pro, nextcloud client 3.6.2), as soon as I open it and start syncing it will do the same thing until the systems are unusable and require a reboot.

arch-user-france1 commented 1 year ago

confirm on macos M2

definitely thrashing with RAM, using up to 4GB and 99.3% cpu looks like a memory leak, though it manages to free some memory sometimes. doesn't respond to anything anymore. happens while syncing.

gessel commented 1 year ago

My problem comes and goes with syncing a live google earth folder (cute way to keep it synced across devices, but doesn't work). As with similar reported issues, it seems to be an issue with a live directory and my guess is that it happens when the application owner of a synced directory does something to the files of that directory that the client doesn't understand and causes a runaway condition of some sort.

I realize this is a nearly uselessly vague description and bereft of useful debug info, but it seems to afflict "live" local directories like games or earth cache directories. These probably aren't good candidates for sync anyway, but it is possible the owner application detects a change and "fixes" it and then the client detects the change and "fixes" it back and forth, but only the nextcloud client seems to go crazy. This may be due to a more compute/memory intensive check process and might be kicking off check threads faster than they can resolve slowly burning up memory and CPU - max CPU might limit it but if the limit is max memory the result is generally going to be a locked system.

arch-user-france1 commented 1 year ago

I rather think it's just too many files. I tried syncing my nextjs code and the same happened. Nextcloud now gone on my computer... Can't use that. Unreliable.

gessel commented 1 year ago

That could be - I have a directory hierarchy with 157,824 files that I tried to sync and couldn't. It doesn't kill memory though, just winds the fan up to max (CPU load), but .googleearth only has 663 items in it. If Google earth is not running, it syncs fine, but soon after starting google earth, NextCloud Client eats all RAM and the computer is unhappy.

Trance-0 commented 1 year ago

Screenshot_20230211_193251 Nextcloud can always take away all of my memories

brousselle commented 1 year ago

I confirm. I had the exact same behavior. I just ugraded to latest Nextcloud-3.7.4-x86_64.AppImage. And now the programs is not responding anymore as soon as i open google earth.

nextcloud / desktop