search

ryanpcmcquen / image-ultimator

:rice_scene: Make images amazing, recursively and automagically.
https://imgult.github.io
Mozilla Public License 2.0
54 stars 2 forks source link

Memory usage on a huge count of images #4

Closed alpharder closed 8 years ago

alpharder commented 8 years ago

Hello, thanks for the script!

However, when i tried to used in directory containing 14G of images, all 64GB of RAM disappeared :)

Can you please fix it? Looks like all processes start without any wait for previously started processes.

2016-02-20 1 08 30

ryanpcmcquen commented 8 years ago

What OS are you on? Also what version of nice do you have?

alpharder commented 8 years ago

Ubuntu Server 14.04.4

abolshakov@ds01:~$ nice --version nice (GNU coreutils) 8.21

ryanpcmcquen commented 8 years ago

What about all the programs imgult uses (their versions)?

More specifically, that would be:

jpegoptim
mozjpeg <== this is where the 'jpegtran' executable should be coming from
optipng
pngquant
gifsicle
svgo
ryanpcmcquen commented 8 years ago

Judging by the path you have there, it looks like you are using the system jpegtran and not mozjpeg, this is going to make a huge difference in not only performance, but also the size of images produced ...

ryanpcmcquen commented 8 years ago

@alpharder did you ever try installing all the appropriate versions of stuff? It appears another user was able to run imgult successfully against 140GB of images, so your sample should be fine.

Please try the new release as well: https://github.com/ryanpcmcquen/image-ultimator/releases

alpharder commented 8 years ago

@ryanpcmcquen We've used another solution that worked well. Thanks!

ryanpcmcquen commented 8 years ago

Glad to hear you found something! What did you end up using?

alpharder commented 8 years ago

This one: https://github.com/caiguanhao/imgcrush

BTW, there was installed mozjpeg, not a jpegtran. I've even created symlink for mozjpeg binary named as jpegtran

ryanpcmcquen commented 8 years ago

@alpharder isn't the mozjpeg binary called jpegtran?

alpharder commented 8 years ago

@ryanpcmcquen When I compiled it from source there was mozjpeg binary generated.

ryanpcmcquen commented 8 years ago

I wonder if the old version Ubuntu ships of pngquant was slowing things down (it doesn't even support the --skip-if-larger argument).

Torkiliuz commented 8 years ago

If you have pngquant 2.0.1 (September 2013), which is what gets installed by default in Ubuntu, you're going to have problems optimizing big folders in one run. A combination of updating pngquant to 2.3.0 (July 2014) and imgult to the latest version 4.0.01 will help tremendously :wink: