Verify ParallelConversions change

[kino13]

Hello,

could someone verify if the ParallelConversions modification has had any effect?

I have been trying every nightly release of the CE edition, and a few days ago I noticed the ParallelConversions in the changelog.

I did some tests converting big files and monitoring my cpu usage and I can confirm that all cores showed an increased usage, and a big speed increase in the conversions as well.

However, I tried to repeat the test yesterday and the CPU's didn't show more than 7 cores being used, and the speed was the usual. I did not keep a copy of the version when I first tested it (my fault), so could someone else verify if they notice any changes?

[maforget]

It might just be a placebo effect. The Default Value was 32, but capped at 8, so most people would have 8 conversion at a time (pages). The change doesn't per say force the number of cores, there is a Parallel.For function used and it just ups the number of MAX parallel operations from 8 to Number of Cores (with max 32). Also for someone that has only 8 cores, it will not change anything, since it be the same. And for people with less than 8 cores, it might even be slower, since the max was 8 before, it could go down on some old hardware. Also if you have any changes to the ComicRack.ini file, it will use that value (again with a max of #cores, before it was 8).

It might be related to the book, if it's an easy to convert, it might not use all the available cores. But if the conversion is slower than it might use them more. It's the framework/the Windows scheduler that seems to decide.

I haven't done a benchmark test at all, but I did find it faster. Might be more of an effect when converting to WEBP (like in syncing optimize) than a straight CBR to CBZ. We could do a comparison, by converting the same book, with the ini value set to 8 and the default of 32.

[kino13]

Thank you for your answer, I am aware that my observations are not very scientific. Is just that this was an aspect of the original comicrack that I found anoying. A bit more of info: I have a 24 core cpu, ALL my conversions are done to webp(level 70). And you are totally right on both aspects, that it may have been placebo, and that it may differ depending on the book being converted, but I swear I saw the percentages of the conversion increasing in 40% each step instead of the usual 2-3%. I am talking of books of +700Mb each here. I tried to repeat the process today, but honestly, it was the same, the cpu usage seemed to be stuck at 5-7 cores. I will keep checking (although I have cleared all my current queue of comics at the moment). As always, thank you a lot for all your work.

On Mon, 12 Feb 2024 at 21:14, maforget @.***> wrote:

It might just be a placebo effect. The Default Value was 32, but capped at 8, so most people would have 8 conversion at a time (pages). The change doesn't per say force the number of cores, there is a Parallel.For function used and it just ups the number of MAX parallel operations from 8 to Number of Cores. Also for someone that has only 8 cores, it will not change anything, since it be the same. And for people with less than 8 cores, it might even be slower, since the max was 8 before, it could go down on some old hardware. Also if you have any changes to the ComicRack.ini file, it will use that value (again with a max of #cores, before it was 8).

It might be related to the book, if it's an easy to convert, it might not use all the available cores. But if the conversion is slower than it might use them more. It's the framework/the Windows scheduler that seems to decide.

I haven't done a benchmark test at all, but I did find it faster. Might be more of an effect when converting to WEBP (like in syncing optimize) than a straight CBR to CBZ.

— Reply to this email directly, view it on GitHub https://github.com/maforget/ComicRackCE/issues/31#issuecomment-1939496340, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADRZHU6B6DLKETAJEM5OIM3YTJZZPAVCNFSM6AAAAABDENOM66VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMZZGQ4TMMZUGA . You are receiving this because you authored the thread.Message ID: @.***>

[maforget]

I did a bunch of benchmarks, and found that there is a speed boost (I get pretty much 100% CPU, from around 75%) when doing a big conversion like JPG => WEBP. But any other conversion where there is no conversion of the file type, doesn't change anything.

There is also something weird when converting to CBZ an already existing WEBP, the CPU usage is very low and the file size doubles (after just the first time).

Results are in a wiki page https://github.com/maforget/ComicRackCE/wiki/ParallelConversions-benchmarks