maforget / ComicRackCE

A Community Edition for the legendary Comic Book Manager ComicRack. ComicRack is back from the dead.
GNU General Public License v2.0
213 stars 20 forks source link

performance issue #38

Closed pingu6 closed 4 months ago

pingu6 commented 4 months ago

Describe the bug ComicRackCE become unresponsive for few minutes

Exact Steps to Reproduce Steps to reproduce the behavior:

  1. add a reading list
  2. select a large number of comics (about 1993 in this case)

Screenshots Taskmgr_pJubc1XfWi

Version/Commit (check the about page, next to the version, for the string between brackets):

Additional context when selecting a fewer number (300) it become unresponsive only for few seconds

maforget commented 4 months ago

Something else is going on here. When you import the list, do you have all the files in the database already? It looks like these are only fileless comics, do you have some actual comics you are selecting?

That list only contains 300 comics, and I was able to select them all without being unresponsive at all. I even went into one of my folder and was able to Select All (Ctrl+A) more than 4000 without any delay (program is running on a NVME, but comics are on a HDD). I even tried with a fresh DB, added a folder to it, scrolled fast in the list and all I could see was thumbnails being created as I was scrolling.

I would expect some kind of slow down when it's actually reading the contents of files but not browsing fileless comics. For example when you go into the folder tabs and the folder has a couple of comics into that folder it takes a while for it to scan all the folder. But there is a menu showing you the progress. Even when it is scanning for new books or creating thumbnails for those new books there is an indicator at the bottom right.

To have these kind of CPU usage, there must be something else that is going on in the background and not just selecting files. You are either exporting, scanning a folder, it is creating thumbnails or syncing at the same time. The only time I feel the program is really sluggish is when I am syncing to my tablet at the same time. With some recent changes to exporting, it can use more processor power and even cap the CPU usage.

Have you modified the ComicRack.ini file to change some of the settings that would affect the program working in the background? Are the files located locally or maybe on some network storage? What are your computer specs? Number of processor threads? Kind of storage?

Simply selecting files shouldn't be using that amount of CPU usage, something else is happening at the same time. Probably creating thumbnails for newly imported books, but even then it shouldn't make the window unresponsive.

pingu6 commented 4 months ago

Something else is going on here. When you import the list, do you have all the files in the database already? It looks like these are only fileless comics, do you have some actual comics you are selecting?

I have some of them but the reading list wasn't sorted and so i used smartlist to only select fileless comics to delete them from the database

That list only contains 300 comics, and I was able to select them all without being unresponsive at all. I even went into one of my folder and was able to Select All (Ctrl+A) more than 4000 without any delay (program is running on a NVME, but comics are on a HDD). I even tried with a fresh DB, added a folder to it, scrolled fast in the list and all I could see was thumbnails being created as I was scrolling.

I didn't want to re import all of that so i only imported that list to test it and screenshot it anyway i re imported all of them and tested it again and i think it took about 5 min image

I would expect some kind of slow down when it's actually reading the contents of files but not browsing fileless comics. For example when you go into the folder tabs and the folder has a couple of comics into that folder it takes a while for it to scan all the folder. But there is a menu showing you the progress. Even when it is scanning for new books or creating thumbnails for those new books there is an indicator at the bottom right.

To have these kind of CPU usage, there must be something else that is going on in the background and not just selecting files. You are either exporting, scanning a folder, it is creating thumbnails or syncing at the same time. The only time I feel the program is really sluggish is when I am syncing to my tablet at the same time. With some recent changes to exporting, it can use more processor power and even cap the CPU usage.

It wasn't doing anything else as far as i know, also after it become responsive again the CPU usage is only about 1%

Have you modified the ComicRack.ini file to change some of the settings that would affect the program working in the background? Are the files located locally or maybe on some network storage? What are your computer specs? Number of processor threads? Kind of storage?

No, i didn't modify anything and yeah they are located locally

CPU: AMD A8-3520M
CPU Cores: 4 / 4 (Number of CPU cores / threads)
CPU Base Clock: 2300.0 MHz
Memory Size: 6 GB
Memory Type: DDR3 SDRAM
Memory Speed: 666.7 MHz (DDR3-1333 / PC3-10600)
OS: Microsoft Windows 10 Professional (x64) Build 19045.4046 (22H2)
Drive Model: Samsung SSD 860 QVO 1TB

Simply selecting files shouldn't be using that amount of CPU usage, something else is happening at the same time. Probably creating thumbnails for newly imported books, but even then it shouldn't make the window unresponsive.

maforget commented 4 months ago

When I say something else is going on, I don't necessarily mean you doing something else in the background, but the program doing something. Check the bottom right for an indicator of a background task, double clicking on it will give you tell you what is doing what. Right now your screenshot always hide that.

If I can't reproduce an issue, I can fix a potential bug. The problem might be your computer, just 4 threads and only 6 GB of memory is very low. Maybe try editing the ComicRack.ini to lower the MaximumQueueThreads & ParallelConversions to 2, so your entire CPU isn't use.

Also specifics are important, you say that it becomes unresponsive when selecting only fileless comics. The bottom says that there is 10GB selected so it isn't all fileless comics. But are you using Ctrl+A to select all or are you dragging the mouse so the slow down would be you going through the list and not the selecting. Or are you selecting 1 and going to the end and Shift clicking? Is it starting when finding a real comic in the list. Have you tried in detail mode, does it do the same? What about existing comics that are already cached in your library, does it do the same also.

pingu6 commented 4 months ago

When I say something else is going on, I don't necessarily mean you doing something else in the background, but the program doing something. Check the bottom right for an indicator of a background task, double clicking on it will give you tell you what is doing what. Right now your screenshot always hide that.

Well I'm doing something else in the background (not that much) but i meant that ComicRackCE isn't doing anything else, also sorry i didn't know about the indicator

If I can't reproduce an issue, I can fix a potential bug. The problem might be your computer, just 4 threads and only 6 GB of memory is very low. Maybe try editing the ComicRack.ini to lower the MaximumQueueThreads & ParallelConversions to 2, so your entire CPU isn't use.

Yeah i think it's about 11 years old? but i don't know, i mean they are just fileless comics anyway editing the ComicRack.ini didn't help

Also specifics are important, you say that it becomes unresponsive when selecting only fileless comics. The bottom says that there is 10GB selected so it isn't all fileless comics. But are you using Ctrl+A to select all or are you dragging the mouse so the slow down would be you going through the list and not the selecting. Or are you selecting 1 and going to the end and Shift clicking? Is it starting when finding a real comic in the list. Have you tried in detail mode, does it do the same? What about existing comics that are already cached in your library, does it do the same also.

Like i said i used a smartlist (it was deleted in that screenshot but i used it when i deleted the fileless comic again and it's the same):

Name "FileLess"
Match [File Directory] equals ""

image image

Yeah I'm using Ctrl+A to select all and detail mode doesn't help, i only have 154 comics but selecting all of them works fine

edit: after i tested and all i was going to delete the fileless comics but this time it wasn't unresponsive at all even when selecting everything, so i deleted them and restarted ComicRackCE then i added them again but it was this same thing so i also edited the ComicRack.ini back but i can't reproduce the issue anymore

I don't know what changed but i also reproduce the issue yesterday also i restart ComicRackCE most of the time when i was reproducing it and there wasn't any background task

maforget commented 4 months ago

Just a FYI when editing the ini, you also need to remove the ; in front to enable the change. The MaximumQueueThread affects how many thread are used for background work, so lowering would leave some room for other windows task and maybe prevent that.

Also for a smart list checking fileless you should use IsLinked instead.