Open Coresi7 opened 2 years ago
NoCodeJustPancakes
commented
2 years ago Same issue here, I made no changes other than upgrading program version. 4.3.9 checks files at 400MB/s+ I switch to 4.4.0 check same file, speeds drop to 100MB/s. Reinstall 4.3.9, check file, back to 400MB/s+,
Cjaker
commented
2 years ago Slow check on 4.4.0 and hangs/freeze hardly (feeling of 1~5 FPS on UI) Downgraded to 4.3.9 and everything is fine
Windows 10 x64 21H2 Build - 19044.1466
UnviableFriend
commented
2 years ago I'm not sure if this is the cause, but I was just doing a force recheck on a 38GB torrent, and I have 32GB of RAM. I noticed my client became very sluggish and looked over to my task manager to see my memory usage was at 100%. Stopping the recheck completely(just a pause didn't work) and my memory was freed. When I tried again I could see my memory usage climbing as the recheck progressed. Restarting qBittorrent didn't fix the issue and memory usage again kept climbing trying to recheck the torrent. Windows 10 qBittorrent: 4.4.0 Qt: 5.15.2 Libtorrent: 2.0.5.0 Boost: 1.78.0 OpenSSL: 1.1.1m zlib: 1.2.11
captincorpse
commented
2 years ago Same issue here. All I did was update to 4.4.0 and the file recheck is extremely slow now. There may be a memory leak, several times my ram maxed out and qBittorent was using it all. Win 10 with 16Gb ram, qBittorrent was using using 9Gb at one time during file recheck. This started when I noticed a torrent wasn't updating file progress during download. It downloaded over 100Gb of a 150Gb file and showed about 9% done, initiated file recheck and it has been running for 2 hours trying to recheck.
thalieht
commented
2 years ago Is qBittorrent v4.4.0 with Libtorrent 1.2.15 still slower than qBitorrent v4.3.9?
Which can be found here on the bottom https://github.com/qbittorrent/qBittorrent/actions/runs/1665307409 It will probably be back to normal with this version because rechecking is a libtorrent operation.
Desef
commented
2 years ago The GUI keeps freezing for 2 to 3 seconds with 4.4.0. Reverting to 4.3.9 has zero freezing. Re-loading 4.4.0 again causes constant freezing.
linax3-genuine
commented
2 years ago just verified a 12GB torrent. the hard disk usage was very bad. hdd sounded bad, run at 100% and was doing only reading at very low speed. everything related to that drive was disturbed. 16gb ram almost filled by the process.
...after some testing, reverted to 4.3.9, all the semi-verified torrents verified normally, while seeding multiple torrents. no ram bloating. hdd works and sounds normally, did not even got to 100%, id guess.
KorbenDls
commented
2 years ago Seeing the same behavior on my system as well on 4.4.1. When checking a torrent, it was maxing out at 7 MB/s. Once I rolled back to 4.3.9 (keeping the same configuration), it's now checking at 220-230 MB/s.
blackbeard1080
commented
2 years ago Seeing the same behavior on my system as well on 4.4.1. When checking a torrent, it was maxing out at 7 MB/s. Once I rolled back to 4.3.9 (keeping the same configuration), it's now checking at 220-230 MB/s.
Same on 4.4.2 rolled back to 4.3.9
ghost
commented
2 years ago Can you reproduce this on windows with 4.4.2?
linax3-genuine
commented
2 years ago im done updating, until this debacle is long gone. i read other issues and the developers seem to have no clue, of what they are doing.
Orhideous
commented
2 years ago Also stumbled upon this issue. Was able to reproduce it with 4.4.0, 4.4.1, 4.4.2 versions (libtorrent 2.x) Changing the number of hashing threads did not bring any results.
How can I help debug this problem?
tan-wei
commented
2 years ago Also for 4.4.2, quite slow for checking a complete torrent.
tan-wei
commented
2 years ago Also stumbled upon this issue. Was able to reproduce it with 4.4.0, 4.4.1, 4.4.2 versions (libtorrent 2.x) Changing the number of hashing threads did not bring any results.
How can I help debug this problem?
I've test 4.3.9~4.4.2, I can confirm it is OK with 4.3.9, but quite slow with other versions.
Orhideous
commented
2 years ago Compiled qbittorrent-nox from master in two flavors — with libtorrent 1.x and 2.x
Tested on 25G torrent. 1.x version takes ≈7s to rehash, 2.x takes 12-13s. Looks like a problem with libtorrent, not qBittorrent.
@PriitUring
ghost
commented
2 years ago @arvidn ping!
linax3-genuine
commented
2 years ago Orhideous, it would be nice, if you do the same test on a good old hardware, like what we, mortals use :). such tests make any flaws quite obvious.
microka
commented
2 years ago I just update to 4.4.2 from 4.3.9, obviously 4.4.2 file checking speed is much more slower than 4.3.9, the files checked are saved on SSD. So disappointing that 4.4.x are still so much bugs, i am thinking whether should i roll back to 4.3.9.
microka
commented
2 years ago @PriitUring Here is some infomation: OS: Windows 11 Pro 21H2 x64 (22000.613) RAM: 16GB*2 + 8GB*1 Files: A folder contains 54 files, each one is 1.x GB and total are 72.4 GB. Drive Type: SSD
qbittorrent_4.3.9_x64_setup.exe + libtorrent 1.2.14.0 (Default settings): 700+MB/s qbittorrent_4.4.2_x64_setup.exe + libtorrent 2.0.5.0 + Physical memory (RAM) usage limit: 512MB (Default Value): 310+MB/s qbittorrent_4.4.2_x64_setup.exe + libtorrent 2.0.5.0 + Physical memory (RAM) usage limit: 16384MB : 380+MB/s qbittorrent_4.4.2_x64_setup.exe + libtorrent 2.0.5.0 + Physical memory (RAM) usage limit: 32768MB : 380+MB/s qbittorrent_4.4.2_RC_1_2_qt5_x64_setup.exe + libtorrent 1.2.15.0 + Physical memory (RAM) usage limit: 512MB (Default Value): 700+MB/s
I think libtorrent 2.x is the cause of the problem.
microka
commented
2 years ago The tests i ran above is extract the setup .exe file using 7-zip, and create a 'profile' folder to run qBittorrent with default settings in portable mode, each one test only 1 single torrent.
v4.3.9 (Qt 5.15.2 + libtorrent 1.2.14.0) vs v4.4.2 (Qt 5.15.2 + libtorrent 2.0.5.0), the two test results I'have already posted. But i don't know how to run the 'latest v4.4.x branch build with Qt5 and libtorrent 1.2.15+' in portable mode.
v4.3.9 with Qt5 and libtorrent 1.2.14+ vs v4.4.2 and latest v4.4.x with Qt5 and libtorrent 2.0.5+ except latest v4.4.x, other results already posted.
I don't know how to get "latest GitHub v4.4.x branch build of qBittorrent with libtorrent 2.0.5+"
microka
commented
2 years ago Is there any progress?
NoCodeJustPancakes
commented
2 years ago Easiest fix: Download 4.3.9 and disable 'Check for program updates' ¯(°_o)/¯
microka
commented
2 years ago Yep! I had downgraded to v4.3.9, everything works fine.
tan-wei
commented
2 years ago Quite slow. 400 torrents will cost 2 days. But for v4.3.9, several hours are enough.
arvidn
commented
2 years ago do the libtorrent-1.2.x and libtorrent-2.0.x tests use the same number of threads to hash with? (this is the aio_threads setting).
If it's hard to tell, when looking at activity monitor, does one peg more CPU cores than the other?
arvidn
commented
2 years ago so maybe fewer aio_threads would be better in 2.0.x. it would probably increase "sequentiality"
Orhideous
commented
2 years ago «New versions of qBittorrent are very slow in checking files!» «It's broken, fix it!» «The developers themselves don't know what's going on...» «Don't use the new versions.»
Enough.
There have been a lot of complaints like this. It's time to investigate and fix this problem. Rolling back to old versions is not a good option, because turning a blind eye won't make the bug go anywhere. As much as one might wish that to happen.
We need to figure out if there is a problem at all, if there is, which component in «libtorrent-qBittorrent-hardware» chain is causing it, and how to solve it.
So... test time!
I installed Kubuntu 22.04 LTS in a virtual machine, installed all the build dependencies, and then cloned the base of that image into three instances.
master branch (commit 9351f66c) compiled qBittorrent 4.5.0alpha1 with libtorrent-rasterbar 1.2.16master branch (commit 9351f66c) compiled qBittorrent 4.5.0alpha1 with libtorrent-rasterbar 2.0.6Compilation instructions are taken from CI workflow, GUI=ON version was compiled.
A 32G payload.bin file was created from /dev/urandom as test data. This file was placed on two 40G virtual disks, NVMe and HDD; v1 torrent was created for the file, 16384 parts of 2M.
Before each test time measurement, the caches were reset both on the host and in the virtual machine, echo 3 > /proc/sys/vm/drop_caches
Each value was obtained by averaging 4 measurements. For the clients, all settings are left default except two, aio_threads and hashing_threads.
Numbers in tables below represents seconds taken to recheck single torrent, if not specified otherwise.
First, the base values on an «old good qBittorrent» in qBt_0, i.e. 4.3.9, with all default settings.
aio_threads=10| Cores | HDD | NVMe |
|---|---|---|
| 1 | 319±2 | 57±1 |
| 4 | 320±2 | 32±1 |
| 12 | 358±3 | 31±1 |
Let's test the first hypothesis that the qBittorrent update caused the slow scanning in some way. To do this, let's compare 4.3.9 and 4.5.0 on the same version of libtorrent-rasterbar, 1.2.16
aio_threads=10| Cores | 4.3.9 | 4.5.0 | 4.5.0 (aio_threads=1) |
|---|---|---|---|
| 1 | 319±2 | 325±3 | 322±2 |
| 4 | 320±2 | 325±2 | 322±2 |
| 12 | 358±3 | 358±1 | — |
aio_threads=10| Cores | 4.3.9 | 4.5.0 | 4.5.0 (aio_threads=1) |
|---|---|---|---|
| 1 | 57±1 | 60±1 | 60±2 |
| 4 | 32±2 | 58±1 | 57±1 |
| 12 | 31±1 | 33±1 | — |
According to the results of the tests, there is no difference in performance between the versions of qBittorrent itself.
Let's check the second hypothesis: maybe the performance problems are caused by some regression in libtorrent-rasterbar.
aio means aio_threads, hashing means hashing_threads |
Cores | 1.x aio=1 |
1.x aio=10 |
2.x aio=1 hashing=1 |
2.x aio=1 hashing=10 |
|---|---|---|---|---|---|
| 1 | 60±1 | 60±1 | 60±1 | 59±1 | |
| 4 | 57±2 | 58±1 | 53±2 | 32±2 | |
| 12 | — | 33±1 | — | 22±1 |
It turned out that libtorrent had nothing to do with it either. But wait a minute…
Let's test it again — but with spinning rust this time!
| Cores | 1 | 4 |
|---|---|---|
1.x aio=1 |
322±2 | 322±2 |
1.x aio=10 |
325±3 | 325±2 |
2.x aio=1 hashing=1 |
337±1 | 322±1 |
2.x aio=1 hashing=2 |
496±3 | 390±1 |
2.x aio=1 hashing=4 |
807±1 | 804±2 |
2.x aio=1 hashing=10 |
916±3 | 868±1 |
Finally.
IOPS starvation, that's it. In SSD-baked setups with plenty of IOPS the problem does not arise at all. The situation changes dramatically on HDDs, especially consumer range HDDs, especially with data fragmentation. Once upon a time @arvidn wrote a note:
It seems HDD performance drops significantly at > 2 hasher threads, whereas SSD performance (which is most likely CPU bound for the most part) just improves the more threads thrown at it.
That's exactly what's happening. Performance significantly drops even at 2 hasher threads. In short, this is a fairly obvious thing described in libtorrent's documentation.
The hasher threads do not only compute hashes, but also perform the read from disk. On storage optimal for sequential access, such as hard drives, this setting should probably be set to 1.
Set Hashing threads value to 1.
Set Hashing threads value to 1 unless you are absolutely sure you know what you are doing! Values greater than 1 can increase performance only in two cases:
I propose to change the default number of hashing threads to 1, instead of 2, as was done in #13499. It may be worth migrating in the configuration and forcing this value to be overwritten, so that at least the user experience is not degraded. /cc @FranciscoPombal @bershanskiy
linax3-genuine
commented
2 years ago interesting post, but im not convinced, that you have found the issue. a lot of code has been changed in libtorrent 2. and hdds behave abnormally, ist not about time-to-finish-a-job. that is expected behavior, as you say yourself.
arvidn
commented
2 years ago @linax3-genuine can you describe the issues you experience when you set hashing_threads to 1?
ghost
commented
2 years ago @Orhideous could you test this as well by setting the hashing threads to zero and see the outcome? https://github.com/qbittorrent/qBittorrent/pull/16956
Orhideous
commented
2 years ago @summerqB Done.
qBittorrent 4.5.0alpha1, libtorrent-rasterbar 2.0.6, 1 core, hashing_threads=0, HDD |
aio_threads |
Seconds |
|---|---|---|
| 1 | 319 | |
| 2 | 480 | |
| 3 | 720 | |
| 4 | 900 |
linax3-genuine
commented
2 years ago @arvidn i can not do testing. IceLancerSR, says it fixes the abnormal behavior, maybe it did. im a bit confused from the names of those things and it seems to me that qbt/libt is overriding os and drivers, which is odd.
tan-wei
commented
2 years ago I try to set aio_threads from 10 to 1, the hashing process is extremely slow. For about 700 torrents, it will take at least 3 days to rehashing. But for 4.3.9, at most one night will finish.
Orhideous
commented
2 years ago @tan-wei Leave aio_threads with default value (10), but instead set hashing_threads to 1.
ghost
commented
2 years ago @Coresi7 can you test with setting the hashing threads to 1? If that fixes the problem for you then I’d consider this issue fixed by #16951 If you see minor improvements only then perhaps there’s some other issue as well. I would request you to restart the PC in between testing with both versions and timing the checking times with a stopwatch and not based on your feelings.
Coresi7
commented
2 years ago @Coresi7 can you test with setting the hashing threads to 1? If that fixes the problem for you then I’d consider this issue fixed by #16951 If you see minor improvements only then perhaps there’s some other issue as well. I would request you to restart the PC in between testing with both versions and timing the checking times with a stopwatch and not based on your feelings.
I'll look into it and try again.
Coresi7
commented
2 years ago @Coresi7 can you test with setting the hashing threads to 1? If that fixes the problem for you then I’d consider this issue fixed by #16951 If you see minor improvements only then perhaps there’s some other issue as well. I would request you to restart the PC in between testing with both versions and timing the checking times with a stopwatch and not based on your feelings.
yes, if i change hashing thread to 1 and i will get significant speed boost. about 20MB/s to 100MB/s. I'm using 4.4.2 now.
Coresi7
commented
2 years ago @Coresi7 can you test with setting the hashing threads to 1? If that fixes the problem for you then I’d consider this issue fixed by #16951 If you see minor improvements only then perhaps there’s some other issue as well. I would request you to restart the PC in between testing with both versions and timing the checking times with a stopwatch and not based on your feelings.
before and after changing speed status in taskmgr.
tan-wei
commented
2 years ago @tan-wei Leave
aio_threadswith default value (10), but instead sethashing_threadsto 1.
OK, I'll try. Thanks.
Update: Yes, it works, now hashing is much faster.
ghost
commented
2 years ago Ok then I consider this fixed by https://github.com/qbittorrent/qBittorrent/pull/16951
linax3-genuine
commented
2 years ago did we learn what went wrong, or we got a lucky guess, that mitigated the issue?
arvidn
commented
2 years ago I'm experimenting with a facility to pick settings based on download path: https://github.com/arvidn/libtorrent/pull/6850
abelletti
commented
2 years ago This issue is absolutely not fixed by reducing the hashing_threads count from 2 to 1, at least in my case.
Situation: Storage is mounted via SMB on a 2014 Mac Mini. Server is ZFS running on Ubuntu 22.04 LTS. The pool in question is served from SSD. It can do hundreds of MB/sec over 10GbE to my other desktops. On the old Mini with 1GbE, 900+ mbps is easily achieved.
Had upgraded to 4.4.2 and some time later noticed that checking was very slow. Went through all the usual things, including reducing the number of hashing_threads as described here. No impact. Not small impact, but none. Check performance consistently showed about 130-140 mbps.
Reverted to 4.3.9 and immediately am checking at 900+ mbps again.
Perhaps the hashing_threads fix helps for local HDD but not remote file systems?
NoCodeJustPancakes
commented
2 years ago Same here, I tried the current stable version, 4.4.2 (not libtorrent 1.2). Set hashing threads to 1.
Same slow recheck speed as when I posted in January.
Downloads go to an SSD, Qbitorrent is running in a VM, drive shares come via SMB, but it's one system. Normal transfers can reach 500MB/s.
I'm back to 4.3.9 again as well. Working as expected.
Pentaphon
commented
2 years ago Ok then I consider this fixed by #16951
I just tried both and there's still a big difference between 4.3.9 and 4.4.3.
4.3.9 is very responsive, even on an older laptop with an HDD, while 4.4.3 is still fairly unresponsive on the same laptop.
You can even tell the difference, but not as much, on a desktop with an SSD.
I would re-open this issue and keep it open until they have the same responsiveness.
arvidn
commented
2 years ago @Pentaphon when you say "responsive", are you talking about checking throughtput? Or are you talking about the latency of something?
in 4.3.9, do you peg more than one CPU while checking? in 4.4.3, do you peg at least one CPU while checking? And do you use 1 hashing thread? I believe the default changed, but perhaps your setting didn't change.
Pentaphon
commented
2 years ago @Pentaphon when you say "responsive", are you talking about checking throughtput? Or are you talking about the latency of something?
Pretty much everything is less responsive on 4.4.3 compared to 4.3.9. You can clearly see the 4.4.3 UI freeze on a laptop with an HDD while 4.3.9 UI doesn't freeze up at all on the same laptop.
Checking torrents: 4.3.9 immediately begins to check while 4.4.3 brings up the "not responding" message on the Windows titlebar for a while before it starts checking. I peg more than 1 CPU while checking on 4.3.9 and 4.4.3.
Starting downloads: I just tried to download Ubuntu 22.04 on both clients and 4.3.9's download bar started filling up immediately while 4.4.3 would take 10-15 seconds before the download bar would start to fill up.
I never changed any thread settings on either.
stalkerok
commented
2 years ago File checking speed for 4.4.0 is much slower than 4.3.9
As well as creating torrents, so that's probably related too. https://github.com/qbittorrent/qBittorrent/issues/16762 @arvidn Please test the speed of creating torrents v1 with the libtorrent 1.x and 2.x libraries.
ghost
commented
2 years ago For people commenting about 4.4.3 please first check if you’ve set your hashing threads to 1. Also setting it to 1 probably won’t fix your issue as there seem to be some downsides of using mmap IO on some storage systems. And the overall unresponsiveness could be due to other factors such as too low working set size or disk cache not flushing in a timely manner. All these other issues have been discussed already in other threads so let’s not bring topics like responsiveness to this thread and keep it strictly to hashing speed.
linax3-genuine
commented
2 years ago so this issue is "strictly for hashing speed". i did not knew that! you are like the doctors, who cure symptoms instead of the illness. 👏
thanks for the boos of my previous comment. i expect the same again.
qBittorrent & operating system versions
qBittorrent: 4.3.9 x64 Operating system: Windows server 2022 datacenter 21h1 (10.0.20348) libtorrent-rasterbar: 1.2.14 qt: 5.15.2
What is the problem?
qbittorrent v4.4.0 with qt6 file check speed is extremely slow, compare to qbittorrent v4.3.9. About 2x slower for hdd, or 10x slower for samba protocol. require for a fix, thanks!
Steps to reproduce
No response
Additional context
No response
Log(s) & preferences file(s)
No response