search

qbittorrent / qBittorrent

qBittorrent BitTorrent client
https://www.qbittorrent.org
Other
28.26k stars 3.97k forks source link

GUI performance with many thousands of torrents - transfer list, icons, and more #13304

Closed FranciscoPombal closed 2 years ago

FranciscoPombal commented 4 years ago

qBittorrent's GUI performance with many torrents (as in, many thousands of torrents) could probably be better.

I created this issue for the purposes of experimentation/investigation/discussion about this, without having to deal with excessive baggage from previous threads.

Relevant background:

TL;DR: a bug in libtorrent was causing a long-running operation to block everything else. While that has been fixed in libtorrent 1.2.8+ (report confirming: https://github.com/qbittorrent/qBittorrent/issues/12825#issuecomment-671676613), the GUI performance still leaves something to be desired.

TL;DR: On Linux, investigation into the problem showed that inefficient WMs/compositors (or some inefficient way qBittorrent interacts with them!) play a significant role in the perceived lack of performance. Other than that, it was probably part https://github.com/qbittorrent/qBittorrent/issues/12825, part https://github.com/qbittorrent/qBittorrent/issues/11822#issuecomment-648335028:

I even ran a small test with callgrind on Xubuntu - not a very thorough one, I did not even have debug symbols (I was using the qBittorrent executable from the official PPA). Even without symbols, I could see that most of the activity came from internal Qt GUI-related stuff, and an enormous amount of calls to somewhere within libX11.so. A lot of calls seemed to be to something related to/with a name like XSetImage/XGetImage - I wonder if this has to do with status icons of each element in the [transfer] list?

This is not a thread for regular bug reports. It is expected that posters have some idea of where the problem might be or have done some preliminary investigation/analysis/benchmarks about the problem with developer/power-user performance profiling tools. This is the place to share such findings.

When posting in this thread and investigating these performance issues, please provide as much information as possible from the get-go, to make it easier for others to understand your setup ad environment.. At least:

The goal is to identify specific points where performance can be improved in relation to the GUI, and particularly in the transfer list. As each specific cause/bug becomes clearer and defined as its own separate matter, discussion about it may move to a dedicated thread about it (this is also useful for having something specific to close when submitting a PR). This one is just for initial investigations/brainstorming, e.g. "What if excessive icon load/unload is slowing down the transfer list?", "How can we test that?" "We could try doing foo..." "I tried \<this patch>, and got the following results..."...

--- Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/92685933-gui-performance-with-many-thousands-of-torrents-transfer-list-icons-and-more?utm_campaign=plugin&utm_content=tracker%2F298524&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F298524&utm_medium=issues&utm_source=github).
xavier2k6 commented 4 years ago

speedplotview assumes speed is updated per second but the default value is 1500ms this causes lots of duplicate data in the calculation for the graph.

May have some relevance?!

PR 13263

jagannatharjun commented 4 years ago

The only thing I can think that is regularly called and can cause this is transfer list refreshing, you can change the interval in the advance setting. Actually, in 4.2.5, subsequent refresh calls can overlap (fixed in master) when CPU is busy causing more damage, so setting large refresh intervals actually may help. AFAIK you can also disable speed graphs from advance settings if you think that is the problem. Another possible reason can be that libtorrent is posting a large number of alerts for some reason, you can add some logging and take a look.

Also, I don't think Qt can be the reason. Qt is very efficient and is used in many other high-end applications So it must be something within qbittorrent itself.

FranciscoPombal commented 4 years ago

@jagannatharjun

The only thing I can think that is regularly called and can cause this is transfer list refreshing, you can change the interval in the advance setting. Actually, in 4.2.5, subsequent refresh calls can overlap (fixed in master) when CPU is busy causing more damage, so setting large refresh intervals actually may help.

Is the transfer list all refreshed at the same time? Is it possible to perhaps optimize it by just refreshing the visual representation of the items that are visible at any given time? Or do we already do that?

As mentioned in https://github.com/qbittorrent/qBittorrent/issues/11822#issuecomment-648335028, could it also be the case that there is room for optimization in the way the status icons are displayed in the list? My suspicion is that a bunch of svg icons frequently being loaded could be a problem, perhaps.

Another possible reason can be that libtorrent is posting a large number of alerts for some reason, you can add some logging and take a look.

A bit of anecdotal evidence: I have noticed, particularly in low end machines, that qBittorrent's WebUI becomes unusable (unresponsive) under high resource consumption scenarios. I suspect this is due to the fact that all parts of the application are fighting unconstrained for a share of the computing resources. Ideally, computations related to GUI code, WebAPI requests (such as those coming from the WebUI), etc should have much higher priority/precedence than, say, libtorrent piece hashing, so that the user can still control the application, even at the expense of a little transfer performance while doing so. I have no idea how this could be implemented, though.

Also, I don't think Qt can be the reason. Qt is very efficient and is used in many other high-end applications So it must be something within qbittorrent itself.

Indeed.

xavier2k6 commented 4 years ago

Perhaps qBittorrent defaults need to be adjusted again?!

Still, bumping from 4 -> 8 would be a good change that would benefit SSD users without hurting HDD users

https://github.com/qbittorrent/qBittorrent/issues/11461#issuecomment-663826350

@nokti can you change your file pool size in advanced settings from 40 to 4000 please & report back. I can confirm I had this very same issue with 4.2.5 and following these instructions seems to have fixed it

https://github.com/qbittorrent/qBittorrent/issues/12914#issuecomment-664022272

In relation to progress bar would the use of palette colors like the pieces bars PR https://github.com/qbittorrent/qBittorrent/pull/13256 have much of an impact or perhaps changing to a different style......QfusionStyle or equivalent (apologies. I don't know which style is used etc)

Another thing, that may have an impact is if torrents are checking on start-up....currently if a torrent is checked/re-checked, there is no record of this being carried out in the execution logs.

jagannatharjun commented 4 years ago

Is the transfer list all refreshed at the same time? Is it possible to perhaps optimize it by just refreshing the visual representation of the items that are visible at any given time? Or do we already do that?

They do update at the same time but the real implementation of torrent update in libtorrent is actually asynchronous and shouldn't block GUI, even with 10000 torrent I don't see it causing too much load to block GUI indirectly.

I think rather then speculating and wasting time it would be better if someone can replicate and profile the application.

could it also be the case that there is room for optimization in the way the status icons are displayed in the list? My suspicion is that a bunch of svg icons frequently being loaded could be a problem, perhaps

Icons are cached.

FranciscoPombal commented 4 years ago

@jagannatharjun

They do update at the same time but the real implementation of torrent update in libtorrent is actually asynchronous and shouldn't block GUI, even with 10000 torrent I don't see it causing too much load to block GUI indirectly.

Ok, not a problem then.

I think rather then speculating and wasting time it would be better if someone can replicate and profile the application.

Icons are cached.

Well said. An idea could be comparing two profiling runs, one with transfer list icons, and one with no transfer list icons at all, just to see how much CPU the icon display code eats up. I can probably do that in the next few days, if no one does it before that.

jagannatharjun commented 4 years ago

one with transfer list icons, and one with no transfer list icons at all

transfer list icons are cached, they can't be the problem, they are only loaded once, but you can try benchmark over it if you wish. almost all the icons loaded in GUI are cached.

xavier2k6 commented 4 years ago

This may also help https://github.com/qbittorrent/qBittorrent/pull/13316

xavier2k6 commented 4 years ago

off topic, where does qBittorrent make use of zlib?

FranciscoPombal commented 4 years ago

@xavier2k6

https://github.com/qbittorrent/qBittorrent/search?l=C%2B%2B&q=zlib

Keep in mind the results preview in GitHub's search GUI only shows the first couple matches in each file.

reyaz006 commented 4 years ago

I'm very sorry but my problem is very similar and was never fully resolved. I'm seeking help and if anyone wants, I can perform tests or benchmarks etc.

The most serious problem is RAM consumption. Sometimes it can stay adequate (2-3 GB) for weeks, and sometimes it's starting to eat more and more by a minute, and when it reaches >20 GB other apps start crashing. My thoughts (and a suggestion) about the issue are here https://github.com/qbittorrent/qBittorrent/issues/11397

I noticed once again that no matter what, on startup qBT seems to access downloaded files of paused and finished torrents, even though there should be no such need. Also noticed that at the startup some torrents (few hundreds) are getting rechecked and switched to "Missing files". It takes hours until it finishes this with all the torrents. Was any of these ever fixed?

Another suggestion about GUI performance is about tags (or categories). I noticed that whenever current list holds smaller amount of torrents, it performs much better. By default the GUI loads always in "All" category, so it always forcing the worst scenario of RAM consumption from the very start. As soon as it becomes responsive I switch to another category. My guess is that if it's possible to make qBT start with an empty category, or a category with just a few torrents, the GUI would load much faster.

I'm not sure what I should provide if you ask me for "Full qBittorrent logs and settings in plain text". My log files are always huge (several MB) and quite useless. Also unicode torrent names are not properly saved. The settings file include some bits that I'd like to keep private, e.g. cookies and paths.

reyaz006 commented 4 years ago

Right now it appears to be working well and responsive but RAM usage is constantly increasing. From 3 GB ~3 hours ago to 7.5 GB now. I didn't do any actions with existing torrents and didn't add any new ones. 23 MB downloaded and 5 GB uploaded.

FranciscoPombal commented 4 years ago

@reyaz006

That's a very old version and a whole lot of torrents indeed. I would suggest backing up all your settings/fastresumes (just in case, see https://github.com/qbittorrent/qBittorrent/wiki/Frequently-Asked-Questions#Where_does_qBittorrent_save_its_settings for more info), and trying a build of recent master commits + libtorrent >=1.2.10, such as the most recent of these: https://github.com/qbittorrent/qBittorrent/actions targeting master.

Many important fixes have been deployed since 4.1.9, including fixes that massively improve memory usage and GUI startup time in some circumstances. Recently a user with many torrents reported a massive improvement with recent commits: https://github.com/qbittorrent/qBittorrent/issues/12825

FranciscoPombal commented 4 years ago

Some general news: https://github.com/qbittorrent/qBittorrent/pull/13348 has been submitted with some improvements to startup code. Furthermore, an issue has been identified with tracker announces that can impact GUI startup time massively. Investigation is still ongoing.

reyaz006 commented 4 years ago

I will try that but which file should I use? What's the difference?

qBittorrent-nightly-e16aafb_vcpkg-fdac1fc_qt5-lts-5.15.0_libtorrent-release-1.2.10_x64-static-release
qBittorrent-nightly-e16aafb_vcpkg-fdac1fc_qt5-lts-5.15.0_libtorrent-RC_1_2-head-aa3faed_x64-static-release

Also will backing up settings/fastresumes really help to avoid massive rechecking in case I downgrade back?

FranciscoPombal commented 4 years ago

@reyaz006

I will try that but which file should I use? What's the difference?

No real difference for our purposes here. Just use qBittorrent-nightly-e16aafb_vcpkg-fdac1fc_qt5-lts-5.15.0_libtorrent-RC_1_2-head-aa3faed_x64-static-release.

Also will backing up settings/fastresumes really help to avoid massive rechecking in case I downgrade back?

Yes.

reyaz006 commented 4 years ago

Just use qBittorrent-nightly-e16aafb_vcpkg-fdac1fc_qt5-lts-5.15.0_libtorrent-RC_1_2-head-aa3faed_x64-static-release.

I've tried this. My observations:

I'm already used to confusion about reading tons of files it wouldn't do anything about (paused torrents will not be downloaded and will not be uploaded). But again I'd like to know if there are any plans to fix this behavior.

FranciscoPombal commented 4 years ago

@reyaz006

Just use qBittorrent-nightly-e16aafb_vcpkg-fdac1fc_qt5-lts-5.15.0_libtorrent-RC_1_2-head-aa3faed_x64-static-release.

I've tried this. My observations:

  • It started to recheck a lot of torrents. >2000 I think and still going. I don't want it to do that.

This may be the result of a "first start" with a version >= 4.2.0 when coming from a previous version. The fastresume file format changed between versions, so it is expected that some rechecks/fastresume re-saving takes place.

From the news section of the website (version 4.2.0):

ATTENTION: This release uses the libtorrent 1.2.x series. It saves fastresumes a bit differently than the 1.1.x series, which are used so far in the previous versions. If you run it and then downgrade to a previous qBittorrent version then your torrents will probably start rechecking.

This is also why I advised you to backup your fastresumes before trying out the new version - in case something catastrophic happened and you had to roll back, it would save you a lot of time rechecking to roll back fastresume formats.

So, it would be interesting to know if this behaviour is reproducible across more than one startup/full close cycle. Assuming you always let qBittorrent exit cleanly between runs (i.e., don't force terminate it), I wouldn't expect those same >2000 torrents to keep rechecking every time.

  • It started to download torrents that were never started. There were added torrents with various settings that should have told qBT to not do anything with them, e.g. magnet links with unticked "Download" or .torrent with all files disabled. Some of those (or all) were started. Downloaded ~20 GB while I waited for the GUI. I don't want it to do that.

It is possible that this is the result of changing over to the new .fastresume format.

It is also possible that there is some other problem with these torrents. If you experience frequent rechecks/redownloads on startup on Windows, even after making sure that qBittorrent has had the opportunity to save the fastresume files in the correct format and exit cleanly, it is usually due to one of these reasons:

* GUI becomes responsive after ~1 hour. Not much better vs. v4.1.9.1.

It is possible all the rechecks and fastresume conversions in this "fresh start" played a role in this, I would expect the startup time to have decreased, all other factors being equal. At least it's nice to now that it's not worse than before. Recently, there have also been some recent developments related to improving startup time in https://github.com/qbittorrent/qBittorrent/pull/13348

  • RAM seems not leaking, or at least not yet. Good.

:+1:

I'm already used to confusion about reading tons of files it wouldn't do anything about (paused torrents will not be downloaded and will not be uploaded). But again I'd like to know if there are any plans to fix this behavior.

  • Immediately it started to read a lot of files related to finished, already seeded or otherwise paused torrents. I don't want it to do that.

If you mean the .fastresumes, it is inevitable for qBittorrent to read those, even for torrents that are stopped. Otherwise, it can't know the information it must show in the transfer list. That being said, it is possible that there is much room for improvement in the code related to reading fastresume files on startup.

reyaz006 commented 4 years ago

So, it would be interesting to know if this behaviour is reproducible across more than one startup/full close cycle. Assuming you always let qBittorrent exit cleanly between runs (i.e., don't force terminate it), I wouldn't expect those same >2000 torrents to keep rechecking every time.

Okay trying it second time as I'm typing this. And yes, I do not terminate qBT process. Just choose Exit and wait for it to close properly.

Notes about the first run:

The GUI loaded now and I didn't notice much of improvement in loading time - only about 10 minutes less. And those >2000 torrents are getting rechecked again. So it will happen again on each application startup.

If you mean the .fastresumes, it is inevitable for qBittorrent to read those, even for torrents that are stopped.

No, .fastresumes are about 500 MB in total, it wouldn't take long to read them. I mean reading hundreds of GBs of downloaded files across my download folders.

FranciscoPombal commented 4 years ago

@reyaz006

There were 2 oddly errored torrents which I was unable to read because neither GUI (status column specifically) nor log file support unicode I guess. One of those was properly downloaded before.

qBittorrent in general most certainly supports Unicode. What is your OS? Are you running a recent Windows 10 version? Are your locale settings correctly configured? It is also possible the fastresume files for those particular torrents are corrupted. It would be nice if you could open a separate issue report for this. If possible, upload both torrents in a .zip, so that others can test on their systems to rule out the possibility of some problem/misconfiguration on your end. But first let's wait for the results of the second run.

There were a lot of "Errored" torrents but those are normal due to missing files etc. There were few marked with something like "Error: Missing file or folder". But in some of those I could clearly see that the file reported in log was not even checked for download.

Do you have .unwanted folders? This feature was buggy and has been removed, it is possible this problem relates to that. It is also possible this particular torrent ended up with an invalid .parts files location in its fastresume. Again, let's see if the problem reoccurs in the second run.

After a long wait there was only 1 torrent stuck at "Checking" and it did not change this status no matter what. It is a fully downloaded torrent from months ago - all files are there.

Try resuming and pausing, and force rechecking, and see if the problem persists in the next startup. If it doesn't work, you can always remove it and re-add it.

Inside the BT_backup folder, there are only ~2700 updated .fastresume files (other remained untouched by 4.3.0a1). Total amount of files is ~76k (38k torrents total).

Strange, I believe all should be touched on exit, or every X (60, I believe) minutes, even if they remained paused during the session. Other than that there are a couple of actions that will force the .fastresume to be overwritten. Resuming/pausing is one such action, I believe. Just to be sure, you can leave the client running for a couple of hours and see if all .fastresumes are updated then.

xavier2k6 commented 4 years ago

What is your OS?

I believe @reyaz006 is still using windows 8.1 x64 as per https://github.com/qbittorrent/qBittorrent/issues/11397

perhaps - pc specs/settings could be looked at to speed up the rechecking....SSD/HDD etc.

(quite alot of torrents there!)

reyaz006 commented 4 years ago

qBittorrent in general most certainly supports Unicode. What is your OS? Are you running a recent Windows 10 version? Are your locale settings correctly configured?

8.1 x64, and most certainly there are no problems with my locale. But I see that there are only question marks in qbittorrent logs for every unicode character in torrent names. They are displayed and operated just fine in qBT list, yes. But something seems off when qBT tries to (a) display a non-Latin OS-reported error in the list and (b) write unicode data to logs.

Do you have .unwanted folders?

I do but I don't think there were such folders for those torrents. Should I try removing all of .unwanted and .parts files?

Strange, I believe all should be touched on exit, or every X (60, I believe) minutes

I set it to 480 minutes because at some point I lost an SSD to qBT. https://github.com/qbittorrent/qBittorrent/issues/9152 I think it's adequate because I don't want to stress my drive that frequently and it should still write everything it touched on proper exit. I'll see how it goes next time.

perhaps - pc specs/settings could be looked at to speed up the rechecking....SSD/HDD etc.

There is no way rechecking speed of tons of GBs can be improved. I think removing the need to recheck that much data is the only good option...

FranciscoPombal commented 4 years ago

@reyaz006

8.1 x64, and most certainly there are no problems with my locale. But I see that there are only question marks in qbittorrent logs for every unicode character in torrent names. They are displayed and operated just fine in qBT list, yes. But something seems off when qBT tries to (a) display a non-Latin OS-reported error in the list and (b) write unicode data to logs.

Echoing my request from the previous post, please submit a new issue report with example torrents where you see this happening.

I do but I don't think there were such folders for those torrents. Should I try removing all of .unwanted and .parts files?

The .part files are needed for torrents where you have deselected some files. If you delete the .part files, they won't recheck to 100%. Come to think of it, part of the problem may be that the fastresumes have. If you are willing to redownload a bit of data, you can try deleting all of your .part files and .unwanted folders. This new qBittorrent version has better .part file handling and doesn't create .unwanted folders. But IMO, you should only resort to this after trying to solve the problem by removing and re-adding the affected torrents - this may save you having to redownload those pieces.

I set it to 480 minutes because at some point I lost an SSD to qBT. #9152 I think it's adequate because I don't want to stress my drive that frequently and it should still write everything it touched on proper exit. I'll see how it goes next time.

The conclusion from that discussion was that .fastresume writes were most likely not the cause of death of your SSD, as I recall. Regardless, that was back when the save interval defaulted to 3 min. The new default has been 60 min for some time, which is 20 times lower - more than adequate. 480 minutes is overkill.

reyaz006 commented 4 years ago

I'm still sorting out the the changes that came with the first run of new qBT version. I'm seeing that in many cases empty folders I created myself were deleted. E.g. there was a fully downloaded / finished / paused torrent with 1 folder and 3 files inside of it. After it was completed I created a subfolder inside it with specific name, so the torrent folder had 1 subfolder and 3 files. Now the subfolder is gone. It seems to me it was really removed by qBT, since I'm keeping backups of folder and file structures and there was no such differences before.

So it looks like qBT started to treat folders of downloaded files as exclusive territory. Is this a thing? Could it be related to handling of outdated .unwanted subfolders?

Echoing my request from the previous post, please submit a new issue report with example torrents where you see this happening.

I re-added the affected torrents and will see if it happens again.

The conclusion from that discussion was that .fastresume writes were most likely not the cause of death of your SSD, as I recall. Regardless, that was back when the save interval defaulted to 3 min. The new default has been 60 min for some time, which is 20 times lower - more than adequate. 480 minutes is overkill.

I've tested with SSD monitoring tool and changing that interval massively affected the amount of data written to disk each 24h. I don't agree with any long interval being an overkill because it really doesn't affect anything (but short interval is an overkill for write-sensitive hardware). On my PC, qBT is running for weeks and months with probably only couple of dozens restarts per a year. Why would I want to write fastresume data frequently if it's being kept in RAM already? With 8h interval, in case qBT crashes, I'm most likely going to rebuild any lost metadata about recently added or downloaded torrents without problem.

reyaz006 commented 4 years ago

After running for more than 24h, I restarted qBT again.

FranciscoPombal commented 4 years ago

@reyaz006

After running for more than 24h, I restarted qBT again.

  • I think there are no more odd errors after re-adding problematic torrents.

Including no more weird text/Unicode errors? Nice, that's good to hear.

  • Same ~2000 torrents are still being rechecked all over again.

  • Again, no more than ~2700 fastresume files were updated. It appears that new version is not supposed to update all fastresume files, or it doesn't do that for some reason.

Hmm, I don't think this is suppose to happen. IIRC we still update .fastresumes even for stopped torrents every fastresume save interval. @glassez can you please confirm this?

Have you let it running long enough for at least a .fastresume save interval to occur? In your case, it would have to be over 8 hours (I would do 24 or a bit more just to be safe). Don't you see related errors in the log? E.g. "file size mismatch" and such

Can you please open a new issue and post a couple of fastresumes + their corresponding torrents for the torrent that are stuck in that recheck loop plus and a couple of fastresumes + their corresponding torrents for torrents that are not affected by the issue? My suspicion is that they will be quite different and the cause of this issue will be exposed by that difference. Alternatively, you can remove them and add them all again... but that is probably quite tedious.

  • Startup time (from launch to GUI becoming responsive) is still about 45 minutes. Not different from 2nd run since the upgrade.

I'm interested in knowing what the startup time will be once all other outstanding issues are resolved (right now, that would be the recheck loop issue + the fastresumes not updated issue, which is likely related to the former).

FranciscoPombal commented 4 years ago

@reyaz006

Recently, there have also been some recent developments related to improving startup time in #13348

So far this endeavour has lead to the fixing of a bug in libtorrent that was causing massive increases in startup time if torrents had trackers with URLs not ending in /announce.

So please test with the build linked in https://github.com/qbittorrent/qBittorrent/pull/13348#issuecomment-692384245. It would be interesting to see whether or not you see an improvement in startup time with these new build, even without having solved the recheck problem as mentioned above.

reyaz006 commented 4 years ago

@FranciscoPombal

So please test with the build linked in #13348 (comment). It would be interesting to see whether or not you see an improvement in startup time with these new build, even without having solved the recheck problem as mentioned above.

I tried this first. Observations vs. qBittorrent-nightly-e16aafb_vcpkg-fdac1fc_qt5-lts-5.15.0_libtorrent-RC_1_2-head-aa3faed_x64-static-release:

Can you please open a new issue and post a couple of fastresumes + their corresponding torrents for the torrent that are stuck in that recheck loop plus and a couple of fastresumes + their corresponding torrents for torrents that are not affected by the issue? My suspicion is that they will be quite different and the cause of this issue will be exposed by that difference. Alternatively, you can remove them and add them all again... but that is probably quite tedious.

So I identified 2 different torrents.

Should I upload .torrent and .fastresume files related to both torrents into new issue?

FranciscoPombal commented 4 years ago

@reyaz006

Should I upload .torrent and .fastresume files related to both torrents into new issue?

Yes, please.

As for the rest of your findings, I would suggest the following: backup the fastresumes of the problematic torrents, and remove them from their original location. The goal is to try to get a clean startup shutdown where no torrents enter an error state/recheck loop. Then we can fairly judge possible advancements made in startup performance. The errored fastresumes are a separate issue most likely, that should be handled separately.

reyaz006 commented 4 years ago

Done.

As for the rest of your findings, I would suggest the following: backup the fastresumes of the problematic torrents, and remove them from their original location. The goal is to try to get a clean startup shutdown where no torrents enter an error state/recheck loop. Then we can fairly judge possible advancements made in startup performance. The errored fastresumes are a separate issue most likely, that should be handled separately.

Sorry but trying to do anything with that amount of torrents (files) is very problematic. Maybe it would be possible if I could somehow export a list of folder paths from selected torrents.

FranciscoPombal commented 4 years ago

@reyaz006

Sorry but trying to do anything with that amount of torrents (files) is very problematic.

No problem I completely understand. Your collaboration and diligence is much appreciated.

Maybe it would be possible if I could somehow export a list of folder paths from selected torrents.

It's a bit unclear to me what you have in mind, but if you want to extract paths, you can probably do that with the WebAPI and some scripting: https://github.com/qbittorrent/qBittorrent/wiki/WebUI-API-(qBittorrent-4.1)

Look into the torrent list and torrent properties API methods.

Pentaphon commented 4 years ago

Honestly, I get the GUI to freeze with just a few hundred torrents. This is the #1 issue that needs to be addressed.

reyaz006 commented 3 years ago

Here is my report after upgrading to 4.3.1 - https://github.com/qbittorrent/qBittorrent/issues/13416#issuecomment-743149308

Okay I decided to just upgrade to 4.3.1.

I see no more issues arise for me regarding random re-checking.

One annoyance remains - it seems whatever state those errored torrents are stuck at, I'm unable to pause them. They will remain in Errored state unless removed or re-downloaded, while I preferred the ability to pause them so they would no longer stay in Errored state. I'm inclined to think I was correct about being able to set torrents into Pause state where they were in Finished/Completed state, even though there was no visual indication of that state change (it came after I restarted the app and it didn't find the files for such torrents).

Another note, startup time was 50 minutes this time. 40k torrents total.

So, not much change GUI-wise. It's still frozen while starting up.

RAM usage seems abnormally small for now, at ~430 MB.

aminpaks commented 3 years ago

Would someone performance test this version of web ui with a couple of thousand items too?

Read more about the progress here.

puddingsoda commented 3 years ago

I'm getting a really bad lag issue on macOS on 4.3.2. The UI lags a lot when scrolling. I checked my CPU usage and it jumps up to 100% whenever I scroll. This leads me to believe that the app is doing something extremely inefficient (maybe refreshing the scroll view many times when scrolling) resulting in this high CPU usage and intense lag.

This is a little disappointing given that qBit is supposed to be a relatively "lightweight" client but is actually eating up CPU and battery when doing basic scrolling in the app. But I don't blame the developers because I'm sure they would have fixed it if they knew the issue.

puddingsoda commented 3 years ago

I don't know how similar this issue is to #12900, which is specific to Macs, but I am mentioning it here just in case it is related in any way. I will also copy and paste a comment I made on that issue.

The UI will start to lag more the bigger the window size of the torrents. If you decrease the number of torrents that can be seen at one time, for example by clicking on the content pane or by making the window smaller, the lag becomes less harsh.

This is likely due to inefficient rendering of the table views which make up the torrent files in the GUI list. I see a CPU spike of 100% when scrolling in qBittorrent [on Mac]. This should not be happening. When I scroll through a list in any other app, it barely breaks 20% CPU usage. I assume this is an issue in qBittorrent simply because it wasn't coded "natively" for the macOS platform.

@FranciscoPombal If you would like me to perform any sort of tests, I would be happy to do so.

FranciscoPombal commented 3 years ago

@puddingsoda Thanks, but I don't have access to macOS. If you can debug/investigate yourself, if you want. Perhaps your findings would help shine light on other issues for other platforms as well.

sledgehammer999 commented 3 years ago

@FranciscoPombal I temporarily unpinned this in favor of the Poll issue. It was a random decision. If you want you can unpin #9849 and re-pin this.

FranciscoPombal commented 3 years ago

@sledgehammer999

@FranciscoPombal I temporarily unpinned this in favor of the Poll issue. It was a random decision. If you want you can unpin #9849 and re-pin this.

No problem, understandable. I wish GH allowed a few more pinned issues. Anyway, I've been using the Projects tab to track important/frequent issues. In fact, this one should have been there already. Will add now.

pinyangcong commented 3 years ago

Disk cache set to '256MiB'

Protektor-Desura commented 3 years ago

Well not sure if this is correct place to post but I will say that the qBittorrent web GUI fails on me if I get above like 1200-1300 torrents in the queue. I can add more using the API and it will respond to the add no problem. If I let it fall back down in to the 900-1000 range then the web interface will start working again showing torrents and all the information. If I am above 1200-1300 the web interface comes up but it won't show any torrents or upload/download total speeds. Every thing is just blank. I can still use the search tab. So the web interface isn't totally broken but it just doesn't like to handle a large number of torrents.

This is for qBittorrent v4.3.5 Web UI (64-bit) on Linux.