v4.4.5 x64 - Insane memory usage (Windows)

[FiTADiNE]

qBittorrent & operating system versions

qBittorrent: v4.4.5 x64 OS: Windows 10 x64 21H2 10.0.19044 Qt: 6.3.0 Libtorrent: 1.2.17.0 32 GB RAM

What is the problem?

Insane memory leak. 5k torrents. In previous versions there was excessive consumption and chrome/telegram/media(vids, streams) crashes, even with a memory usage limit.

Steps to reproduce

1. Start qB
2. Wait
3. ???
4. Profit.

Additional context

No memory leak with all seeding torrents turned off.

Log(s) & preferences file(s)

No response

[glassez]

In previous versions there was excessive consumption and chrome/telegram/media(vids, streams) crashes, even with a memory usage limit.

Why do you report about previous versions?

[FiTADiNE]

Why do you report about previous versions?

Just for additional information. Main report about last version.

[adem4ik]

Seems like a duplicate of many other issues reported this year. It is connected with libtorrent v2 - https://github.com/arvidn/libtorrent/issues/6667

Speaking of the current topic, it is better to switch to the client version with libtorrent v1. And since 4.4.5 it is default build. You can get under qBittorrent Windows x64 link from https://www.fosshub.com/qBittorrent.html

After installing it you'll see something like this in About dialog:

[FiTADiNE]

@adem4ik, sorry, there was a mistake in libtorrent ver. My current version is 1.2.17.0 already.

[adem4ik]

@FiTADiNE in this case you can limit RAM usage by setting up Tools>Options>Advanced>Disk cache (Кэш диска). -1 means unlimited/auto. Only build with libtorrent v1 has such option.

[FiTADiNE]

@adem4ik, probably, in prev releases there was RAM limit, but now it's disk cache limit. Omg... Thanks, i'll try it.

[ElectricityMachine]

This is a separate thing, but how did you get your task manager to be dark like that?

[FiTADiNE]

@adem4ik, still the same. @ElectricityMachine, just windows dark theme.

[pulbitz]

4.4.5(libtorrent1+qt5) x64 uses 300-400 MB more memory than the disk cache size. 4.3.9 did not.

[FiTADiNE]

@pulbitz, best way is downgrading to 4.3.9?

[pulbitz]

@pulbitz, best way is downgrading to 4.3.9?

I think it's a bit odd to use around 300-400 MB more memory than the disk cache size I specified. That's all. I'm still using 4.4.5 (libtorrent1+qt5) as there are no other inconveniences.

[glassez]

4.4.5(libtorrent1+qt5) x64 uses 300-400 MB more memory than the disk cache size.

Do you think qBittorrent uses memory only for the disk cache?

[pulbitz]

I usually use the disk cache size set to 4096 MB. Below is the result of measuring memory usage in Process Explorer. And I tested it by setting it to 1024 MB.

[FiTADiNE]

Working Set limit 1024MB.

25% of 32GB RAM used after 15 minutes. (Work around application stuttering on Windows not fixed [fixed in 4.4.4]).

Disk Cache limit 1024MB.

68% of 32GB RAM used after 15 minutes and growing. (Peak Working set cuz i've tried 2048MB Disk Cache).

[stalkerok]

FiTADiNE, decide already which version of qbt you are using and on which libtorrent.

[FiTADiNE]

@stalkerok, already decided in 1st post.

[glassez]

Working Set limit 1024MB.

25% of 32GB RAM used after 15 minutes.

There are less than 1GB of RAM used on this screenshot. The similar contradiction is in your second screenshot and related comment.

[FiTADiNE]

@glassez, yes, cuz it's older version on libtorrent 2 with Working set limit parameter and w/o memory leak. Here you can see 2 screenshots: 1066MB qB usage with total 88% of RAM (28.16GB) and 21% total of RAM (6.72GB) w/o qB.

Today i've tested older version, and last on Qt5. 4.4.5 has memory leak with libtorrent 1.2.17.0 on any Qt. 4.4.3.1 on libtorrent 2.0.6.0 doesn't have memory leak, but has problem with "work around application stuttering on Windows" which was fixed in 4.4.4.

Now i'm testing 4.4.5 RC_2 with libtorrent 2.0.7.0

[stalkerok]

@FiTADiNE, have you tried lowering the memory priority?

[glassez]

1066MB qB usage with total 88% of RAM (28.16GB)

Unfortunately, your screenshot does not provide any information about what actually uses all this memory.

4.4.5 has memory leak with libtorrent 1.2.17.0 on any Qt

There can be no memory usage that is not taken into account by the operating system (unless of course there is a memory leak in the operating system itself). Even if there is a memory leak in some application program, the operating system knows that this memory is occupied by this program.

Of course, this memory can be used by the operating system itself for something related to the operation of qBittorrent (for example, for I/O caching). Therefore, it would be much more useful if you presented a more accurate source of memory consumption. Besides, it would be better to see reports in a more structured form (since you decided to compare data on the work of several qBittorrent versions).

[FiTADiNE]

@stalkerok, no, should i? And why?

@glassez, lolwhat? 1st screen: qB works, 88% of 32GB RAM. Second: qB closed, 21% of 32GB RAM. What kind of screenshot do you need? Maybe i should record video? UPD: i'll be glad if you tell me, how can i present more accurate source of memory consumption and structured form of reports.

[stalkerok]

@FiTADiNE, disable the OS cache and set the memory usage to the lowest level, this will reduce the memory consumption.

[glassez]

1st screen: qB works, 88% of 32GB RAM. Second: qB closed, 21% of 32GB RAM. What kind of screenshot do you need?

Where it is seen what exactly uses all these memory.

[glassez]

structured form of reports

Since you provide reports about different qBittorrent version don't forget to attach info about used qBittorrent, libtorrent, OS, etc.

[FiTADiNE]

Where it is seen what exactly uses all these memory.

Okay, where could i see it?

Since you provide reports about different qBittorrent version don't forget to attach info about used qBittorrent, libtorrent, OS, etc.

Already done (with attached screenshots and text information in posts).

v4.4.5 RC_2, Qt 6.3.0, libtorrent 2.0.7.0. 30% of RAM used. Works fine probably, but Private bytes...?

[ghost]

I do not understand what people expect the RAM usage to be with 5000 torrents. If qBt has to constantly access and read files from disk, the OS is definitely caching some of them and even with a working set limit you may not have control over how the OS behaves. If there is a memory leak, you should run a profiler and collect data for the developers to look into it and see if there's indeed a leak or it's just the OS.

[Pentaphon]

I do not understand what people expect the RAM usage to be with 5000 torrents.

What do you expect it to be?

[FiTADiNE]

@summerqB, well, now i'm testing v4.4.5 Qt6 libtorrent 2.0.7.0 and i don't have any problems with RAM. 350MB usage with 1024MB Working set limit, and 21% usage of total RAM with no "Work around application stuttering on Windows".

[ghost]

I do not understand what people expect the RAM usage to be with 5000 torrents.

What do you expect it to be?

Anywhere between 16 to 32 GB. Also someone should not be seeding 5k torrents on a regular PC and expect the torrent client to not bog down the system. It should be done on a dedicated system with adequate CPU, RAM, Network capacity and proper storage medium to be able to handle the I/O demand.

[ghost]

Also you should note that less RAM limit will result in more page faults, the OS will flush the dirty pages earlier, as well as the clean pages, resulting in frequent read/write which could’ve been avoided if the program had cached more data in physical memory.

[NepBlanc]

To note Windows is notoriously bad with caching files in memory. So saying 5k torrents, that is in any sense a lot of data potentially.

I may want to ask, what is the dataset we are talking about? How big is it? How many files? What is the storage medium? iSCSI/SMB? (like is it externally on a NAS or something). Are drivers up to date for the storage controller? Any other hardware information we could get about the system?

For some perspective: OS is Ubuntu22.04 128 GB ECC RAM ryzen 9 3950x Diskset is 6x 18TB WD Gold +2x1TB NVME (L2ARC) The filesystem is ZFS Raid5 With 2x3x18TB so there are 2 loose arrays. qBittorrent-nox Version 4.3.9 in Docker container

I run 2.6k torrents. Dataset is about 30 TB active in qBittorrent. I run my docker qBittorrent with 10-gigabyte ram usage easily if I don't set the memory limits. The kicker is though that ZFS on its own will cache into ARC(RAM)/L2ARC(SSD/NVME), Saying this will resolve some issues with the paging issues vs the issues you get in windows. But this will still consume more ram.

Do note this is not the only task for this machine! Also, I'd never recommend using a machine that is that bogged down as your personal rig. I'd set up a dedicated machine if you go that far.

[LarLei]

I run 2.6k torrents.

Just curious, does it mean 2.6k active (actually downloading and/or seeding) torrents?

I'd assume there is a difference in system resources (i.e. CPU, RAM) between active vs stalled torrents.

[sserdda-liamE]

I do not understand what people expect the RAM usage to be with 5000 torrents.

What do you expect it to be?

Anywhere between 16 to 32 GB. Also someone should not be seeding 5k torrents on a regular PC and expect the torrent client to not bog down the system. It should be done on a dedicated system with adequate CPU, RAM, Network capacity and proper storage medium to be able to handle the I/O demand.

Uh, this is µTorrent 2.2.1 with 2500 and ~20TB seeding at 20mbps. Takes 14 seconds to load. Has never bogged down anything:

I've always found it odd that so many people seem to think that torrenting is some high resource activity that requires a dedicated modern server, when µTorrent is basically invisible on a Pentium 3.

[Pentaphon]

I've always found it odd that so many people seem to think that torrenting is some high resource activity that requires a dedicated modern server, when µTorrent is basically invisible on a Pentium 3.

I don't get it either. They are the same people who recommend a Raspi for torrenting thousands of files which is already done just fine with other clients. The issue is that libtorrent still hasn't become as stable as uTorrent. libtorrent has so many memory leaks that people just assume torrenting even just hundreds of torrents takes several GBs of memory.

[glassez]

libtorrent has so many memory leaks that people just assume torrenting even just hundreds of torrents takes several GBs of memory.

A lot of people don't understand the meaning of the term memory leak at all, so they call it any large memory consumption.

[ghost]

Uh, this is µTorrent 2.2.1 with 2500 and ~20TB seeding at 20mbps. Takes 14 seconds to load. Has never bogged down anything

I wonder why so many people are switching from such wonderful piece of software to an inferior client like qBt. I also wonder why they even bother to comment here 😁

[sserdda-liamE]

Uh, this is µTorrent 2.2.1 with 2500 and ~20TB seeding at 20mbps. Takes 14 seconds to load. Has never bogged down anything

I wonder why so many people are switching from such wonderful piece of software to an inferior client like qBt.

Freetardism is a weird religion. It motivates those caught in its warped ideology to proselytize 24/7 for software based not on any analysis of quality, but merely on the legal licensing scheme it is released under. As normal people do not proselytize software, this causes a minority of fanatics to dominate the volume of traffic. And your average person is biased to believe a suggestion he hears many times on the assumption that it would not be made if it didn't have any backing, as leading him astray would be antisocial behavior that your average person would avoid. Freetardism not only has no internal prohibition against such antisocial behavior, it demands it in the name of the spread of Freetardism. Thus general population is thus convinced to move based on nothing but a false assumption that the behavior they witnessed was sociable.

I'm not sure how people manage to get mentally trapped in such a silly bubble as Freetardism. It seems to me that just an ounce of pragmatism is all that is required to pierce the surface film and see the bubble for what it is. But it seems like the common person has an inordinate desire to belong to a group, causing all lines of intellectual inquiry to be corrupted and coopted into curved lines in the name of fulfilling said desire.

I prefer straight lines. They align with my autistic desire to be objectively correct. They often lie in a messy heap, but there's an underlying, indescribable beauty to it.

https://www.youtube.com/watch?v=2sPDPr3S7Zg

[Pentaphon]

I wonder why so many people are switching from such wonderful piece of software to an inferior client like qBt. I also wonder why they even bother to comment here

Maybe they want to simply point out that qBit is actually far from stable and that uTorrent, despite its bad reputation, remains a stable, low-resource torrent client that is still a benchmark that qBittorrent (more accurately libtorrent) still needs much work to meet or exceed.

[sledgehammer999]

Uh, this is µTorrent 2.2.1 with 2500 and ~20TB seeding at 20mbps. Takes 14 seconds to load

And your average person is biased to believe a suggestion he hears

Your use case certainly isn't that of an average person's use case. You're a power user. Power users should test and see what software best fits them. The average person can make do with almost any torrent client software.

That said, you're more than welcome to report bugs on qbt, help test and possibly provide fixes. Ranting about it gets you nowhere.

[Pentaphon]

Ranting about it gets you nowhere.

To be fair, ranting about the other issues that libtorrent 2.x had finally made you guys decide to (rightfully) go back to libtorrent 1.2.x by default so user rants do actually sometimes get them somewhere that is beneficial to the userbase.

Ignoring what users say is what actually gets a project nowhere, since this project is for end users including yourself, after all.

[Seeker2]

The problem pointed out by FiTADiNE is more of a virtual memory leak than a physical memory leak. I've pointed out this before: qBitTorrent Virtual Memory Leak https://github.com/qbittorrent/qBittorrent/issues/7531

And I've also noted that physical memory usage by qBT's cache was ~30% higher than the cache size was set to, suggesting very poor memory use in indexing the cache and/or physical memory allocations for the cache larger than the cache's actual usage. That among other reasons was why libtorrent v2.x was dropping its own internal cache.

[glassez]

The problem pointed out by FiTADiNE is more of a virtual memory leak than a physical memory leak.

https://github.com/qbittorrent/qBittorrent/issues/17680#issuecomment-1279647263

[Seeker2]

Is your contention that this is not a virtual memory leak or that my terminology is incorrect?

...Because it helps to clearly define the problem, if there is one, before continuing!

[glassez]

Is your contention that this is not a virtual memory leak or that my terminology is incorrect?

My contention that "A lot of people don't understand the meaning of the term memory leak at all". For them "high memory consumption" is equal to "memory leak" which is incorrect. "Memory leak" is a strictly defined problem. And it cannot be identified based only on symptoms in the form of high memory consumption (it may not cause such symptoms at all, while the program's error-free operation may). By using this term, you seem to be claiming that the cause has already been identified and it lies precisely in a "memory leak", and not in some other problem.

[Seeker2]

I claimed the problem the initial poster is reporting is a virtual memory leak NOT a standard physical memory leak or high physical memory usage. Now "working as designed" can be the cause but what is being reported/claimed here are the symptoms.

I mentioned my earlier report of somewhat-higher-than-expected physical memory usage caused by qBT's file cache using 30% more physical memory than the max size of the cache...which is a side issue but perhaps related, since such things do increase virtual memory usage as well.

[glassez]

the problem the initial poster is reporting is a virtual memory leak NOT a standard physical memory leak or high physical memory usage.

As for the "virtual" memory leak, it is generally incorrect to use "virtual" in this context (when we are talking about an application program, considering it from the inside). Virtual memory is exactly what is available to the application program, it does not know anything about physical memory at all (in the sense that it does not use physical memory directly). It is the OS that decides which pages of the virtual memory of a process reside in physical memory at a given time. In any case, you should not claim that this is a memory leak unless you have actually inspected the code (this is practically the only way to detect a memory leak).

[Seeker2]

Would triggering Windows bluescreens suffice?

[Pentaphon]

My contention that "A lot of people don't understand the meaning of the term memory leak at all". For them "high memory consumption" is equal to "memory leak" which is incorrect.

I have never seen people think of those as the same thing. When people report this issue, they know it is a specific problem. They do not think that qBittorrent uses too much memory by its nature and that memory consumption should be lowered. They are saying that qBittorrent suffers from memory leaks and they should be investigated and fixed.

[glassez]

They do not think that qBittorrent uses too much memory by its nature and that memory consumption should be lowered. They are saying that qBittorrent suffers from memory leaks and they should be investigated and fixed.

That's exactly what I'm talking about. You are confusing "the cause of the disease with its symptoms". As I said above, you cannot know that this is a memory leak unless you have actually inspected the code since inspecting the code is practically the only way to detect a memory leak (static analysis can help with some of them).

[Pentaphon]

As I said above, you cannot know that this is a memory leak unless you have actually inspected the code since inspecting the code is practically the only way to detect a memory leak

Inspecting the code is the only way to find the exact cause of a memory leak, but the symptoms of memory leaks get spotted all the time by people who report most or even all their RAM being used for no reason by an exe. It is strange that people seem to think that it is normal for a torrent client to require up to 32GB of RAM for a few thousand torrents when other clients seem to do similar loads with much less than 32GB of RAM.