Nexcloud client memory consumption is too high

[Wenzel]

Hi,

i just noticed that my nextcloud client is using too much memory on my laptop. He is just watching a few files and really shouldn't consume almost 400MB of my RAM.

Details: OS: Fedora 27 nextcloud is installed via package manager: nextcloud-client version: 2.3.3

If anyone has the same issue, please give a feedback here.

Thanks !

[ErikThorsell]

I have the same issue. Nextcloud version: 2.3.3-1, installed from AUR OS: Arch Linux (uname -r == 4.15.15-1-ARCH)

I attributed the issue to me having synced a lot of files recently, but after the syncing was done the high memory usage remained. Heck, Nextcloud is using more than twice the memory of Firefox, that should not need to happen. Right?

[camilasan]

Agreed, we can improve. That should not need to happen.

[Wenzel]

@camilasan can we ping Nextcloud devs to have some idea how to debug this ?

How can we provide you more information and help you find the bug ?

[camilasan]

There are a few things I could think of: clone the repo and debug it on Qt Creator. What were you doing when you first notice the memory usage (syncing, sharing..)? I would also look in the logs based on that info and maybe open the client from the terminal to check for any weird warnings/errors and repeat what you did that might have increased the memory usage. But I won't have time to look at this right now. So yes, please, ping people on irc (#nextcloud-dev / #nextcloud-client), or in the forums.

[affinityv]

This is still a problem 2.5.1.61652 on Windows Server 2012 R2 was using close to 2 GB after being left running for some time.

[bill-mcgonigle]

On Debian Buster, Nextcloud 2.5.1git, I am at 280MB RSS at launch, which rapidly grows to 700MB RSS within a minute or so. After the machine has been up a while I've seen it up at 1.8GB RSS so I usually have to quit it and run Nextcloud occasionally, which I'd rather not do.

I am using pmap -x (pidof nextcloud) where the parens are backticks, but I can't get the markdown escape sequences to work right now.

What is the design target for memory usage?

[Victor-Morel]

On Debian sid, Nextcloud 2.5.2 is using up to 3GB of RAM. I know I have many files, but it is clearly impractical :/

[jpeeters]

Hi,

Thx for developping NextCloud. This is a great tool.

However, I have Nextcloud client running on Arch for 2 days and it consumes more than 12GB of RAM (32GB of RAM available on this machine) and counts dozen of subprocesses/threads.

Is that normal ? Have you already have tried a valgrind run on your code ?

Best, Julien

[janprzy]

The Nextcloud client is currently using 1.55 GB on my Mac, so definitely not exclusive to Linux

[ConstantMown]

Same here, Nextcloud desktop version 2.3.3 on MacOS 10.14.6, uptime about 15-16 days, memory usage is about 1.4 GB, yes the unit is gigabytes. Update to 2.5.3, just after update memory usage was 41.6 MB, unit is megabytes. About four hours after update memory usage is 128.3 MB.

1st edit: In my opinion there is memory leak(s) in desktop client. 2nd edit: After eight hours the memory usage is 218.7 MB. I have not added or removed files. In Finder the size of the managed folder size 45.1 MB and it has 35 "items".

[omni-vi]

Same here as well. MacOS client 2.5.3release (build 20190724). Client started with a memory consumption for just under 50MB - after 4 days it is now at just under 450MB.

[ConstantMown]

I've been running script that runs "heap -s NextCloudClientPID" command once per Nextcloud client process, and then it sleeps for 24 hours.

There has been no new uploads into my Nextcloud, and the number of files and folders is about twenty.

I've attached the two files to this comment Below are some differences between these two files

--- nextcloud.1522.2019-08-27.txt 2019-08-27 09:42:01.000000000 +0300 +++ nextcloud.1522.2019-09-01.txt 2019-09-01 09:42:30.000000000 +0300

-Date/Time: 2019-08-27 09:42:01.355 +0300 +Date/Time: 2019-09-01 09:42:20.719 +0300 Launch Time: 2019-08-27 09:37:49.378 +0300 OS Version: Mac OS X 10.14.6 (18G95) Report Version: 7 Analysis Tool: /Applications/Xcode.app/Contents/Developer/usr/bin/heap Analysis Tool Version: Xcode 10.3 (10G8)

-Physical footprint: 40.8M -Physical footprint (peak): 47.2M +Physical footprint: 2.6G +Physical footprint (peak): 2.6G [snip] -All zones: 186228 nodes (25357696 bytes) +All zones: 14461382 nodes (2732489072 bytes)

nextcloud.1522.2019-08-27.txt nextcloud.1522.2019-09-01.txt

Could @Nextcloud or for example @rullzer take a look at this issue? Q: Why did you pick @rullzer? A: Because he was the last person to update Nextcloud client sources in github.

[bill-mcgonigle]

@ConstantMown : any chance you could bisect the versions? Or run it under Heob or Valgrind? e.g. https://doc.qt.io/qtcreator/creator-heob.html

[ConstantMown]

I'm running MacOS Mohave, version 10.14.6 (18G95). Mohave is even worse than High Sierra, version 10.13, when it comes to "Random Apple operating system developer knows better than the end user, what the end user wants to do" mentality, e.g. Mohave prevents doing bunch of stuff, in an effort to keep end user safe, but I digress.

1. Sorry I don't understand what you mean by "bisect the versions".

2. Downloaded heob sources from github and trying to build it results in

$ make all i686-w64-mingw32-windres -DHEOB_VER_STR=\\"3.2-dev-20190902\\" -DHEOB_VER_NUM=3,2,0,99 -DHEOB_PRERELEASE=1 -DHEOB_COPYRIGHT_YEARS=\\"2014-2019\\" heob-ver.rc heob-ver32.o make: i686-w64-mingw32-windres: No such file or directory make: *** [heob-ver32.o] Error 1

3. trying to run nextcloud in valgrind gives this error message:

$ valgrind /Applications/nextcloud.app/Contents/MacOS/nextcloud ==6365== Memcheck, a memory error detector ==6365== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al. ==6365== Using Valgrind-3.16.0.GIT and LibVEX; rerun with -h for copyright info ==6365== Command: /Applications/nextcloud.app/Contents/MacOS/nextcloud ==6365== --6365-- UNKNOWN mach_msg unhandled MACH_SEND_TRAILER option --6365-- UNKNOWN mach_msg unhandled MACH_SEND_TRAILER option (repeated 2 times) --6365-- UNKNOWN mach_msg unhandled MACH_SEND_TRAILER option (repeated 4 times) ==6365== Thread 2: ==6365== Invalid read of size 4 ==6365== at 0x10CBE2808: ??? (in /usr/lib/system/libsystem_pthread.dylib) ==6365== by 0x10CBE2497: ??? (in /usr/lib/system/libsystem_pthread.dylib) ==6365== by 0x10CBE23FC: ??? (in /usr/lib/system/libsystem_pthread.dylib) ==6365== Address 0x18 is not stack'd, malloc'd or (recently) free'd ==6365== ==6365== ==6365== Process terminating with default action of signal 11 (SIGSEGV) ==6365== Access not within mapped region at address 0x18 ==6365== at 0x10CBE2808: ??? (in /usr/lib/system/libsystem_pthread.dylib) ==6365== by 0x10CBE2497: ??? (in /usr/lib/system/libsystem_pthread.dylib) ==6365== by 0x10CBE23FC: ??? (in /usr/lib/system/libsystempthread.dylib) ==6365== If you believe this happened as a result of a stack ==6365== overflow in your program's main thread (unlikely but ==6365== possible), you can try to increase the size of the ==6365== main thread stack using the --main-stacksize= flag. ==6365== The main thread stack size used in this run was 8388608. --6365:0:schedule VG(sema_down): read returned -4 ==6365== ==6365== HEAP SUMMARY: ==6365== in use at exit: 0 bytes in 0 blocks ==6365== total heap usage: 0 allocs, 0 frees, 0 bytes allocated ==6365== ==6365== All heap blocks were freed -- no leaks are possible ==6365== ==6365== For lists of detected and suppressed errors, rerun with: -s ==6365== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 4988 from 21) Segmentation fault: 11 $

FWIW running "/Applications/nextcloud.app/Contents/MacOS/nextcloud" starts the Nextcloud Desktop app.

I can create a new virtual machine, running a linux desktop of sorts, probably Ubuntu 18.04.3 LTS and try to run Nextcloud desktop in valgrind or Qt5+heob if valgrind route does not work.

[ConstantMown]

Hello @bill-mcgonigle and @nextcloud I ran

valgrind --leak-check=full --show-leak-kinds=all --trace-children=yes --log-file='nextcloud-desktop.valgrind.%p.log' $(which nextcloud)

in a Ubuntu 18.04.3 LTS desktop. I made a new user into my Nextcloud server and used that in the Nextcloud desktop. The desktop client was installed from Ubuntu repository, which was added 'sudo add-apt-repository ppa:nextcloud-devs/client' command. After about ten minutes, I stopped the nextcloud desktop.

The before mentioned command made about 67 megabytes of logs.

$ du -sh *.log
4.0K    nextcloud-desktop.valgrind.1.log
25M nextcloud-desktop.valgrind.2588.log
392K    nextcloud-desktop.valgrind.2612.log
8.0K    nextcloud-desktop.valgrind.2613.log
9.2M    nextcloud-desktop.valgrind.2615.log
9.2M    nextcloud-desktop.valgrind.2641.log
25M nextcloud-desktop.valgrind.2728.log
$ 

The logs have been zipped into the attached file. Attachment size is about 2.5 megabytes. nextcloud-desktop.valgrind.zip

[misch7]

Hey,

we've just released 2.6.1 RC1 which is built with Qt 5.12.5 on all platforms (also has OpenSSL 1.1.1d, so it features TLS 1.3).

You may give it a try: https://github.com/nextcloud/desktop/releases/tag/v2.6.1-rc1

Would be good to know if the problem was caused by the older Qt versions used in the previous builds.

[ConstantMown]

Hi,

I just updated to 2.6.1 RC1 using install package from https://download.nextcloud.com/desktop/prereleases/Mac/Nextcloud-qt5.12.5-2.6.1.20191018rc1.pkg

[ConstantMown]

Hi,

Unfortunately the nextcloud client process shows signs of memory leak.

2019-10-19 11:47 nextcloud 43.5 MB, 7 threads, ports 250 Nextcloud Extensions 16.5 MB, 3 threads, ports 148

2019-10-20 12:54 nextcloud 186.0 MB, 7 threads, ports 268 Nextcloud Extensions 17.0 MB, 3 threads, ports 149 Nextcloud Extensions 16.9 MB, 3 threads, ports 149

On the latter one there were two separate Nextcloud Extension processes.

In my case nextcloud is synchronizing the "nextCloud" folder which contains one 177 MB zip file other files two sub folders.

nextCloud/ 220 MB nextCloud/Documents 135 KB nextCloud/Photos 37 MB

[fuzunspm]

I'm having the same issue on macOS 10.15 and Nextcloud Client 2.5.3release (build 20190724)

Uptime: 5 days, 3 hours, 18 mins nextcloud memory = 3,30 GB -- Threads = 9

[msilvoli]

Hello,

Version 2.6.1 stable (build 20191104) running on Plasma 5 Opensuse Linux 15.1 still showing problems with memory management. 2 days after initial boot, client is consuming around 700MB + memory on swap file. No big sync with the server (all files less than 1MB) . I attached one photo of mem consumed and a more detailed report about the Nextcloud (apprun) memory usage.

nextcloud-client-2.6.1-memory-usage.txt

[sir-brickalot]

Same problem on a MacOS here: After a couple of days unattended run time I usually have around 13GB of memory used by Nextcloud desktop client on MacOS 10.14.6. File system APFS, 2 NC accounts, NC version 2.6.1 stable. Special situation might be that it's a hackintosh. Don't know if this matters.

I appended some screenshots and a log file in the hopes that might help. nextcloud.log.zip

[brianfmorris]

As others have mentioned this is a mac issue as well -

I am using Version 2.6.1stable (build 20191105) memory usage is 4.31GB of 24. I have only a few files in the cloud.

[nicokaiser]

Version 2.6.2rc1 (build 20191209) is still using ~300 MB after some days, with 667 threads.

There still must be some kind of memory leak, seems like 2.6.2rc1 just eats the memory at a slower rate. I will revert to ownCloud client (the one without the QtWebEngine bloatware) and see if it performs better over some days.

[nicokaiser]

With version 2.6.2 (release) it seems to stay quite stable at 7 threads / 70 MB now. I'll observe this further.

[ConstantMown]

I updated in MacOS 10.14.6 nextCloud desktop app to 2.6.2stable (build 20191224). The app was started on December 30th when its memory usage was 44,8 MB. After 48 hours the memory usage was 103,4 MB. Thread count has stayed same all the time 7 threads. IMHO some of the memory leaks were fixed in 2.6.2, but it seems to me that there are one or more memory leaks in 2.6.2. According to app previous activity was six days ago, when I marked one reminder as complete.

[VietTralala]

I also observe a huge memory consumption after a couple of minutes on the current AppImage Nextcloud 2.6.2-x86_46 obtainable via the website for Linux. My OS is Ubuntu 18.04.03. Right after Start: 76MB, 9 Threads After 10 min 1.3GB -1.8GB, >100 Threads

Threads are counted via: ps -o nlwp <pid-of-nextcloud>

EDIT: Nextcloud desktop client: Version 2.6.2stable (build 20191224) Built from Git revision 1d7455 on Dec 24 2019, 12:18:39 using Qt 5.12.5, OpenSSL 1.1.1d 10 Sep 2019

[msilvoli]

Plasma 5 Opensuse Linux 15.1 with client version 2.6.2stable (build 20191224) is much more stable compared with older versions like the comments of @nicokaiser and @ConstantMown. The memory consumption right after start was 41MB and I can see some memory management (going up and down) and almost no swap like my first comment here at this topic. I'll update this comment further with more details.

After 24h running : 82 MB / 10 Threads After 55h running : 170 MB / 10 Theads / 4.1GB disk Read / 2.4GB disk Write. After 73h running : 227 MB / 10 Theads / 4.1GB disk Read / 2.7GB disk Write. After 96h running : 228 MB / 10 Theads / 4.1GB disk Read / 2.7GB disk Write.

[FuturePilot]

2.6.2 on MacOS is definitely an improvement but there still seems to be something leaking. After 3 days it's only gone up to slightly over 300MB. Before it would be at 1GB or more after one day.

The odd thing is I am only seeing this problem on MacOS. On my Kubuntu 19.10 system it never goes above 100MB.

[VietTralala]

TLDR: I attached a valgrind log where I could observe nextcloud hogging memory.

This was my procedure: I build the nextcloud desktop client manually from source by following the wiki instructions

make .. -DCMAKE_INSTALL_PREFIX=~/.local/ -DCMAKE_BUILD_TYPE=Debug -DNO_SHIBBOLETH=1

The info from the running client: Built from Git revision 1d7455 on Jan 4 2020, 14:49:20 using Qt 5.9.5, OpenSSL 1.1.1 11 Sep 2018

I could still observe the memory growth to over 2GB, >200 Threads in around 24h of usage. Next I tried to run valgrind on the compiled binary ~/.local/bin/nextcloud following these instructions. The command was:

G_SLICE=always-malloc G_DEBUG=gc-friendly valgrind -v --tool=memcheck --leak-check=full --num-callers=40 --log-file=valgrind.log ./nextcloud

---

I've let nextcloud run for a couple of hours until the memory growth was above 2GB,>200Threads were running and then manually exited the nextcloud client. The resulting log file is attached:

valgrind.log

I think the interesting part LEAK SUMMARY begins at line 6933. However I miss the expertise to make any sense of it. But if anyone wants more data to debug this issue I am wiling to help.

[ConstantMown]

My Nextcloud client, version 2.6.2stable (build 20191224) in macOS 10.14.6, has been up for fifteen days. The memory usage according to Activity Monitor is 333 MB and 7 threads. Stopping Nextcloud client and killing Nextcloud Extensions manually and then starting nextcloud client, its memory usage is 44 MB.

[ConstantMown]

I updated from 2.6.2stable to 2.6.4stable (build 20200303) week and eight hours ago. The memory usage after start was the same as before ,about 44 MB. Now the memory usage is 189.2 MB. Thread count has not changed since start.

[V13Axel]

I can confirm this as well, with the desktop client taking up to 3GB of RAM within a few minutes.

I've worked around it a bit by running nextcloud with systemd-run --user --scope -p MemoryLimit=500M nextcloud, but of course it would be preferable for the client to not use so much RAM.

[oneilonline]

Nextcloud Linux Client 2.6.4git Ubuntu 19.10

I still consuming a lot of ram, with 6 days uptime the RAM usage was at ~1.5GB, only going back down after a restart. I will try what V13Axel suggested, where and how do I set these? Also I don't see those options in: https://docs.nextcloud.com/desktop/2.6/advancedusage.html ...?

Thank you

[Lucavon]

I have the same problem. I'm using it on Arch Linux, and I have a game client's directory synced using it. The problem starts when I start playing the game - every second, a few lines are added to the log file. Every single time this happens, the file is sent to the cloud - which wouldn't be a problem - but after just 1-2 hours of this, due to a memory leak, the nextcloud client's memory usage is at 4 GB and more!

[olissongs]

Hi, same problem in nc-c 2.6.4git on ubuntu18.04 16 GB RAM, ncc is starting with 400MB in the Morning and freeze Machine with 8 GB in the evening. having a 100GB data in the NC, so it seemes that everything is handled in device-memory.

[benitofischer]

Hi! I have the same problem with nextcloud 2.6.0 on MX Linux. 1,6 GB RAM usage. Is the any solution for that issue?

[christianlupus]

I am running Archlinux and the installed version is 3.0.2. I have a bigger installation but the sync is restricted to "only" ~6000 folders and 240,000 files. So I assume a few MB are well ok for the sync process.

After a few days of letting the sync client run (interrupted with hibernation) I have a process of multiple GB swapped out:

Restarting caused a completely different image: The RAM usage has halved and virtual memory almost gone to an 8th. I then triggered a manual sync for all folders. During the run the RAM was increased (ok, this is expected) and reduced afterwards again. A few MB are swapped in the ending but not multiple GB.

I am having a closer look currently as I was surprised by an almost 10GB heavy swap usage by the client process a few weeks ago (no restart for a few weeks so it cumulated).

[ConstantMown]

I'm currently using Nextcloud client version 3.0.3 (macOS). The client process has been up for about twelve days, and the "nextcloud" process memory usage is about 112 MiB, so memory usage is now about one tenth of what it was when I was running client version 2.3.3.

[er-vin]

Yes, we got a couple of fixes in the 3.0 cycle for memory leaks. Looks like it's starting to pay off.

[AkechiShiro]

Using this version :

Nextcloud version 3.0.1git
Git revision f021db125a0f1f006c4b40d7db0a4b2d0c025de3
Using Qt 5.15.2, built against Qt 5.15.1
Using Qt platform plugin 'xcb'
Using 'OpenSSL 1.1.1h  22 Sep 2020'
Running on Arch Linux, x86_64

I have some issues concerning RAM usage, after 60 hours the process of the nextcloud client uses almost 1GB of RAM.

I'll be updating soon and reporting if the issue is fixed with the latest build from git.

[er-vin]

I'll be updating soon and reporting if the issue is fixed with the latest build from git.

Or at least 3.0.3, some of the fixes I have in mind made it through to 3.0.2 and 3.0.3

[gpothier]

Hi, currently on 3.1.3 on Ubuntu 20.04, memory usage still very high, with 4GB total and 1GB resident: 517037 gpothier 20 0 3979,2m 1,0g 50,2m S 7,6 6,7 35:14.84 nextcloud

[mailinglists35]

memory leak on macos mojave. installed just as a demo, just three files under 1Mb each that nobody uses anyway!. after 11 days uptime, memory consumption in Activity Monitor is 2.11 GB. this causes intense swapping because it's a virtual machine with limited ram

[ConstantMown]

Just to leave another data point. I'm running Nextcloud Desktop Client version 3.2.3 on macOS 10.15.7, 28 days of uptime, memory usage is at 78 MiB.

[hwittenborn]

I was having issues with my Kubuntu 22.04 install freezing, and Nextcloud appears to be the culprit - when I started modifying files in a repository the desktop client spiked up to 3GB of memory usage, though I've since killed the client because it keeps causing issues.

[RedFox134]

Running the windows Nextcloud Desktop Client Version: 3.6.0 OS: Windows 10.0.19.044

Seeing the following and have seen it as high as 17GB ram usage.

Edit: Disabled Virtual Files and it seems this may have fixed the issue. The client isn't locking up any more and I don't see excessive RAM usage any more. Currently syncing a folder(15GB 66673 files) and using 176MB.

Edit 2: Looks like I spoke too soon. Currently sitting at 21GB of RAM usage while I try to sync 306 MB for 18660 files. Virtual files are still turned off.

Edit 3: Seems the functionality causing this is that NextCloud first reads/loads all the files in question before uploading them. If there are looooots of files for it to scan it will load all those into memory and will only start to upload after it's finished scanning the contents of the folder. I think this is intended behavior, but may be a slight bug in that it would be more user friendly to scan one file, upload one file, then move to the next? I don't know if there are issues with this way as well, but it would likely stop the memory run off and the window showing as unresponsive as it scans everything in the folder you're syncing before starting the upload.

[biglari]

I have a similar issue with memory consumption.

As soon as I add a second user to the client, it will over the next 10-30 minutes use all available memory. The queue has only a few hundred megabytes of files to upload.

I have observed the client, and found it starts runaway memory consumption after strange behavior. The progress bar which shows how many bytes have already been transferred suddenly turns into a large negative number, and then quickly adds up again to the actual transferred amount. For example, if it says it uploaded 200 MB, it will suddenly show -552486244 bytes, and then quickly increment the number shown, until it lands back at 200MB. If I restart synchronization at that point, I will prevent large amounts of memory consumption and upload will continue correctly. If I don't respond quickly enough, the program starts to become unresponsive, and memory consumption grows.

I was not able to get a screenshot fast enough as the numbers were negative, but here they are as they go up again:

If I do not restart synchronization fast enough , I also see a lot of Network error 99.

This error has happened every time I add a second user to the client. Each time I added a user who had this error, they uploaded different files from different folders.

[Tom60chat]

Same issue, but he was kind of linked to the windows file explorer freezing after doing a right click.

Edit 1 : In case this is a big amount of files related issues. Size : 483 GB Size on disk : 5,73 GB Content : 142 010 Files, 30 401 Folders

[louisld]

Same memory problem here. The desktop application struggle to sync from client to server, shows negative or wrong number of bytes synced (ex: -2 051 254 out of 651MB). After syncing some files, no more files are synced and memory usage go up to several GB. I then need to kill the process to restart syncing.

After several tries, it finished syncing all files and started to work normally again.

Os: Windows 10 Desktop: Nextcloud Desktop 3.6.4 Server: Nextcloud 25.0.2

[Trance-0]

The nextcloud can always take all of my memory and crash everything. 99% of force restart on my computer is cased by the nextcloud.