search

JacobDev1 / xl-converter

Easy-to-use image converter for modern formats. Supports multithreading, drag 'n drop, and downscaling.
https://codepoems.eu/xl-converter
Other
167 stars 5 forks source link

AVIF | Use all available CPU resources - parallel jobs/workers #20

Closed Shanti099 closed 4 months ago

Shanti099 commented 5 months ago

Hello,

Thank you for your great application, i use it to convert images to Avif, but conversion speed does not use all available CPU resources (Ryzen 3900x) and it fluctuates from 5% to 50%. so average ~25%.

Could you please add a function to select the number of parallel jobs/workers, as implemented in MegaPixel and in MegaPixel-Qt by Alkl58 (also Python language), when converting to Avif. MegaPixel convert images to Avif about ~4 times faster, by using all CPU resources. But, your application better realised and more convenient to use.

Thank you for all your great work.

2024-04-15_185902

JacobDev1 commented 5 months ago

Hello,

this is not a bug.

avifenc encoder is the only one not being run in parallel because it simply crashes when I try to run multiple copies of it. Instead, avifenc receives all available threads and handles multithreading on its own.

XL Converter dynamically adjusts the number of workers, which is why it's so fast. There is no need to do it manually. Changing the number of threads will also impact the number of workers.

For example, when converting 2 images with 8 threads, XL Converter would spawn 2 parallel workers with 4 cores made available to each encoder.

Unfortunately, avifenc is the only encoder not being run this way.

I'll look into researching this further, but I don't promise anything.

Shanti099 commented 5 months ago

Thank you for your reply,

i just tested MegaPixel-Qt by Alkl58 (also Python language) with avifenc [1.0.4] - 2024-02-08 used 12 workers and 24 threads without any crashes. CPU used 100%.

Hope you can find solution, Thank you so much for all your time and work.

JacobDev1 commented 5 months ago

Could you please check for me the version of your encoder with avifenc --version? Does it say aom?

I remember that it only crashed sometimes...

I'll test the latest avifenc in parallel and have a look at the source code of the program you mentioned.

Shanti099 commented 5 months ago

2024-04-15_202750

Shanti099 commented 5 months ago

MegaPixel-Qt had older version of avifenc, i replaced it with avifenc [1.0.4] - 2024-02-08

JacobDev1 commented 5 months ago

Alright, thank you.

I'll post the results of my tests and research on this topic when I'm finished.

JacobDev1 commented 5 months ago

During the initial testing I didn't see any crashes.

If no issue comes up, parallel AVIF encoding will be available in XL Converter 1.0.

Thanks for bringing my attention to this.

Shanti099 commented 5 months ago

Great, Thank you!