Slow processing jpeg-files

[ertyz]

ECT ~2 times slower than Leanify on same jpeg files with same final result. As I can see ECT and Leanify both use mozjpeg module (or maybe even ECT use Leanify, I don't know)

But ECT process jpeg-files 2-2.5 times slower with binary same result.

For comparison I used binary compiled blobs from FileOptimizer 9.8 package on Windows 10 x64 (intel i3-7100, HT on, ssd) ECT 0.7 Leanify 0.4.3

Some guys got same result: https://goo.gl/hXejg9

May be it is problem at this specific binaries, idk. But .png processing look nice (ECT 2-2.5 times faster)

[AlyoshaVasilieva]

Set mode to 1, 2, or 3 (or don't specify, which makes it 3) instead of 9. With mode >= 4 ECT will always try to generate a non-progressive JPEG in case that is smaller. However, for large files it is extremely unlikely to be the case and will just waste time.

For me, ECT -1 -progressive -strip is faster than mozjpeg's jpegtran -copy none

[ertyz]

Without progressive mode operation is fast, yes. But what I want to say, is in both cases ECT and leanify uses "progressive" mode, both keep metadata and both work over mozjpeg. And finally we get binary equivalent files. And anyway ECT work 2-3 times slower.

PS > (Measure-Command {Write-Host ((.\ECT --mt-deflate -p --strict -1 G:\test\jpg\ect1p) | Out-String)}).TotalSeconds
Processed 4 files
Saved 1.58MB out of 12.41MB (12.7567%)

9.539432
--------------------------------------------------
PS > (Measure-Command {Write-Host ((.\ECT --mt-deflate -p --strict -3 G:\test\jpg\ect3p) | Out-String)}).TotalSeconds
Processed 4 files
Saved 1.58MB out of 12.41MB (12.7567%)

9.5062151
--------------------------------------------------
PS > (Measure-Command {Write-Host ((.\ECT --mt-deflate -p --strict -9 G:\test\jpg\ect9p) | Out-String)}).TotalSeconds
Processed 4 files
Saved 1.58MB out of 12.41MB (12.7567%)

12.5015205
--------------------------------------------------
PS > (Measure-Command {Write-Host ((.\Leanify -i 5 --jpeg-keep-all-metadata G:\test\jpg\lean5) | Out-String)}).TotalSeconds
Processing: G:\test\jpg\lean5\20170331_150507_P3310610.JPG
7.04 MB -5.82 MB      Leanified: 1.22 MB (17.32%)
Processing: G:\test\jpg\lean5\IMG_20160719_191416.jpg
1.29 MB -1.23 MB      Leanified: 68.64 KB (5.18%)
Processing: G:\test\jpg\lean5\IMG_20160723_170356.jpg
473.80 KB -441.52 KB  Leanified: 32.28 KB (6.81%)
Processing: G:\test\jpg\lean5\IMG_20170329_194935.jpg
3.62 MB -3.35 MB      Leanified: 271.70 KB (7.34%)

4.7928518
--------------------------------------------------
PS > (Measure-Command {Write-Host ((.\Leanify -i 30 --jpeg-keep-all-metadata G:\test\jpg\lean30) | Out-String)}).TotalSeconds
Processing: G:\test\jpg\lean5\20170331_150507_P3310610.JPG
7.04 MB -5.82 MB      Leanified: 1.22 MB (17.32%)
Processing: G:\test\jpg\lean5\IMG_20160719_191416.jpg
1.29 MB -1.23 MB      Leanified: 68.64 KB (5.18%)
Processing: G:\test\jpg\lean5\IMG_20160723_170356.jpg
473.80 KB -441.52 KB  Leanified: 32.28 KB (6.81%)
Processing: G:\test\jpg\lean5\IMG_20170329_194935.jpg
3.62 MB -3.35 MB      Leanified: 271.70 KB (7.34%)

4.7994808

p.s. without '-p' flag we need only 3.2 seconds, but leanify can't switch off that flag, so I can't make a related comparison

[ertyz]

And as I notice, we always get binary same output jpeg files if flag "--strict" is set, doesn't matter "-1" or "-9" is set. But "-9" always want more time. May be we need to ignore "-1-9" flags if "--strict" mode is set? Should I make it as new issue?

[AlyoshaVasilieva]

It is the binaries in FileOptimizer that are the problem, I think. Using my AVX2 compile of current ECT vs FileOptimizer's Leanify:

PS C:\temp\test> Measure-Command { .\Leanify.exe -i 30 .\out_leanify.jpg }
TotalSeconds      : 1.4554144

PS C:\temp\test> Measure-Command { ect -1 -strip -progressive .\out_ect.jpg }
TotalSeconds      : 1.2696275

PS C:\temp\test> Measure-Command { .\Leanify.exe -i 30 .\leanfolder\ }
TotalSeconds      : 5.7131207

PS C:\temp\test> Measure-Command { ect -1 -strip -progressive .\ectfolder\ }
TotalSeconds      : 5.0195643

From more testing, if I compile ECT with anything less than -msse4 (FileOptimizer is probably SSE2 ECT) JPEG compression time doubles. I don't know why this happens. I don't know if it's bug in compiler or fhanau's code.

[ertyz]

ok, thank you for explanation