Open FranciscoPombal opened 4 years ago
This would be great. I've recently tried having around ~1500 torrents and it's slowing to a crawl and almost unusable. The api request to sync/maindata frequently times out with a net::ERR_CONTENT_LENGTH_MISMATCH 200 (OK) error, sometimes showing the red unreachable message at the bottom left.
I read through #10999 and I don't think those suggestions would fix my (#15067) problem. Full updates are still needed sometimes and they simply don't finish under 30 seconds. To me it sounds like full update performance needs to be fixed first, I don't think serializing some 100 kilobytes of json should take half a minute on any modern processor
Any ideas / eta when this can be addressed. I hit 10K seeds and qbit webui is taking a huge long dump to do anything.
Any ideas / eta when this can be addressed.
awesome, @glassez !!
i have AMD FX-8350 and a bit over 3k torrents, and when i add a few new torrents (and downloading starts happening), the webui is totally unusable until those downloads complete.
do you think it might come in 4.5.3? or is it still pretty early in testing? if you need another tester, i could try.
do you think it might come in 4.5.3?
4.6.0. You can already test if you want https://github.com/qbittorrent/qBittorrent/actions/workflows/ci_windows.yaml?query=branch%3Amaster (first is latest commit).
nice
ok, i will give it a whirl. i'm trying this build script just because i think it's sexy:
docker run -it -w /root -v $HOME/qbt:/root alpine:edge /bin/ash -c 'apk update && apk add bash curl && curl -sL git.io/qbstatic | bash -s all --qbittorrent-master --optimize --icu'
:)
i think it's using libtorrent v2.0.9 because i forgot to configure that. hopefully i don't need master.
ok, here is my unscientific test. took me about an hour.
AMD FX-8350, >3k torrents seeding (different machine running browser accessing WebUI)
maindata
latencymaindata
request latency, session download xfer, session upload xfer, CPUi repeated both tests a couple of times for each setup and got pretty consistent results.
qBittorrent v4.3.9
Qt: 5.15.9
Libtorrent: 1.2.19.0
Boost: 1.82.0
OpenSSL: 3.1.0
zlib: 1.2.13.zlib-ng
idle (uploading @ 3MiB/s):
maindata
latency: ~1500–2000msload @ t=60s:
maindata
latency: could not determineCPU-starved to the point of the WebUI not even loading. couldn't get data.
qBittorrent v4.5.2
Qt: 6.5.0
Libtorrent: 1.2.19.0
Boost: 1.82.0
OpenSSL: 3.1.0
zlib: 1.2.13.zlib-ng
idle (uploading @ 3MiB/s):
maindata
latency: ~700msload @ t=60s:
maindata
latency: typically >2000ms (5000ms or even 10000ms typical)qBittorrent v4.6.0alpha1
Qt: 5.15.9
Libtorrent: 2.0.9.0 <--- note this
Boost: 1.82.0
OpenSSL: 3.1.0
zlib: 1.2.13.zlib-ng
idle (uploading @ 3MiB/s):
maindata
latency: typically <100ms (20ms not uncommon), at most 200ms if i scroll through my long torrent list and start randomly clicking, but such spikes might have been coincidence, idkload @ t=60s:
maindata
latency: <100msamazing web api performance. piss-poor bt performance.
qBittorrent v4.6.0alpha1
Qt: 5.15.9
Libtorrent: 1.2.19.0
Boost: 1.82.0
OpenSSL: 3.1.0
zlib: 1.2.13.zlib-ng
idle (uploading @ 3MiB/s):
maindata
latency: <100msload @ t=60s:
maindata
latency: <100mslibtorrent: v1.2.19.0 >> v2.0.9.0
qbittorrent: 4.5.2 > 4.6
maindata
webapi performance (WebUI)qbittorrent: 4.6 >> 4.5.2 > 4.3.9
btw, all versions tested take ~9s to initially load the webui
this 4.6 maindata
speedup is easily the biggest improvement to qbittorrent in the ~3 years i've been using it. i love it!
Issue created to track progress on implementing the ideas discussed on this thread: https://github.com/qbittorrent/qBittorrent/issues/10999.
Though the original issue was solved, the
sync/maindata
endpoint can still be significantly sped up, by implementing a "torrent history" mechanism.The idea is that each time a sync request is made, the API backend monitors the changes in the state (essentially libtorrent alerts) until the next request. When the next request does come, only the changes need to be serialized, and can be sent right away. This contrasts with the current method, whereby the whole state is serialized each request and the differences to be sent are computed by comparing two full serialization results (the previous one and the current one) each request as well.
The speed up resulting from implementing this change could finally enable qBittorrent to become a "t10k" client (meaning a client capable of reasonably handling 10s of thousands of torrents). Today, even on powerful machines, qBittorrent is not that usable with such high numbers of torrents loaded (talking about headless/WebAPI usage here).