parallel thumbnail generation takes a lot of time to launch a pool instance

[paolobenve]

now in master a parameter can be set in order to choose parallel or cascade processing of thumbnails

If parallel is set (the method is derived from Joachim's code), stating that a thumbnail exists requieres about a fraction of a second, while in cascading mode (derived from paolo code) it takes about 1/10 time

@jallakim could you investigate why?

[paolobenve]

It seems that a long time is needed to launch the parallel process

[joachimtingvold]

Hi,

I'm going to look at it. I also want to implement "the best from both worlds"; combine both parallel and cascade. Something like this; for each "album" (folder), queue all images that needs thumbnails, and after all the files in that folder is done, start processing the queued objects. For each queue (currently n-1, where n=number of cores), do cascade thumbnailing for each image. This would also utilize the cores more efficiently (since the current parallel is limited to the number of thumbnails -- if you have 20 cores, and you only have 150/800/1200/1600px thumbnails, you'd only use 4 cores).

My only concern with cascade, is "JPEG-ification" of the images (i.e. JPEG-artifacts gets worse and worse for each "cascade"). Either way this "redesign" of parallel/cascade would still make it faster than it currently is, even if one disables the cascade (via options/config).

[paolobenve]

The cascade code doesn't jpeg-ificats any image, because, it works with the internally resized image, and doesn't read any jpg thumbnail from disk.

[joachimtingvold]

You still get one more iteration? Not sure if it's visible to the naked eye, though.

Cascade;

Original image --> 1600px 1600px --> 1200px 1600px --> 800px 1600px --> 150px

Non-cascade;

Original image --> 1600px Original image --> 1200px Original image --> 800px Original image --> 150px

[paolobenve]

?

[joachimtingvold]

My point was that you get one extra JPEG-transcode when doing cascade, compared to non-cascade, and in theory this would introduce more JPEG-artifacts.

I'll look into this -- the "combined" mode you already prepared/did should in theory be the fastest, at least once I'm done optimizing it (i.e. figure out why parallel is slow in the first place, and then combine the two).

[paolobenve]

JPEG-transcode happens when the thumbnail is saved. Every thumbnail is saved once, independently of the method used to generate it