search

toy / image_optim

Optimize images using multiple utilities
https://github.com/toy/image_optim
MIT License
1.52k stars 109 forks source link

`check_status!': SIGSEGV (SignalException) #158

Closed srghma closed 6 years ago

srghma commented 6 years ago

can I skip this fatal error?

 0.129089892s pngquant --quality\=0-85 --speed\=3 --output\=/tmp/CaravanersFlyHonduranFlagOnUSborder-wpFPapr30-1820180515-1-1ttqukm.png\~ --force 256 -- ./CaravanersFlyHonduranFlagOnUSborder-wpFPapr30-18.png\~
✓ 0.985957719s advpng --recompress -4 --quiet -- /tmp/Antonio\ Nieto20180515-1-1ye94g9.png
optimizing:  23.6% (ETA: 1.3m)✓ 1.244918716s pngout -k0 -s0 -q -y /tmp/CaravanersFlyHonduranFlagOnUSborder-wpFPapr30-1820180515-1-1ttqukm.png\~ /tmp/CaravanersFlyHonduranFlagOnUSborder-wpFPapr30-1820180515-1-1stqeox.png\~
✓ 1.884116546s advpng --recompress -4 --quiet -- /tmp/CaravanersFlyHonduranFlagOnUSborder-wpFPapr30-1820180515-1-1stqeox.png\~
optimizing:  24.0% (ETA: 1.3m)✗ 5.192978882s pngout -k0 -s0 -q -y ./American\ Affairs.png /tmp/American\ Affairs20180515-1-1gnmra0.png
✓ 2.665579539s advpng --recompress -4 --quiet -- /tmp/American\ Affairs20180515-1-1gnmra0.png
optimizing:  24.3% (elapsed: 31s)
/usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim/cmd.rb:45:in `check_status!': SIGSEGV (SignalException)
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim/cmd.rb:13:in `run'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim/worker.rb:155:in `run_command'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim/worker.rb:127:in `execute'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim/worker/pngout.rb:34:in `optimize'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim.rb:112:in `block (4 levels) in optimize_image'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim/handler.rb:41:in `process'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim.rb:111:in `block (3 levels) in optimize_image'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim.rb:110:in `each'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim.rb:110:in `block (2 levels) in optimize_image'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim/handler.rb:25:in `for'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim.rb:109:in `block in optimize_image'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim/cache.rb:24:in `fetch'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim.rb:108:in `optimize_image'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim.rb:126:in `optimize_image!'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim.rb:246:in `block in run_method_for'
        from /usr/local/bundle/gems/progress-3.4.0/lib/progress/with_progress.rb:101:in `block (3 levels) in run_with_length'
        from /usr/local/bundle/gems/progress-3.4.0/lib/progress.rb:80:in `step'
        from /usr/local/bundle/gems/progress-3.4.0/lib/progress/class_methods.rb:25:in `step'
        from /usr/local/bundle/gems/progress-3.4.0/lib/progress/with_progress.rb:100:in `block (2 levels) in run_with_length'
        from /usr/local/bundle/gems/in_threads-1.5.0/lib/in_threads.rb:290:in `block (4 levels) in runner_thread'
        from /usr/local/bundle/gems/in_threads-1.5.0/lib/in_threads.rb:204:in `catch'
        from /usr/local/bundle/gems/in_threads-1.5.0/lib/in_threads.rb:290:in `block (3 levels) in runner_thread'
        from /usr/local/bundle/gems/in_threads-1.5.0/lib/in_threads.rb:178:in `block (2 levels) in initialize'
toy commented 6 years ago

Adding debugging mode with logging every command before running them would certainly help :) Can you add $stderr << "#{cmd_args.shelljoin}\n" before success = run_command(cmd_args) in worker.rb to see the tool that causes SIGSEGV and also reduce number of threads to 1?

srghma commented 6 years ago

Sorry, I haven't experienced this error anymore, closing this

srghma commented 6 years ago

stumbled to this issue again

Using script 

optimize_images () {
  quality="$1"
  dir="$2"

  echo "Optimizing dir $dir, quality - $quality"

  docker run --rm \
    -v $dir:/usr/src/app \
    -w /usr/src/app ruby:2.4-stretch bash -c \
    "gem install image_optim image_optim_pack && \
    (curl -L \"http://static.jonof.id.au/dl/kenutils/pngout-20150319-linux.tar.gz\" | tar -xz -C /usr/bin --strip-components 2 --wildcards \"*/x86_64/pngout\") && \
    image_optim --no-threads --verbose --allow-lossy --jpegoptim-allow-lossy true --jpegoptim-max-quality $quality --pngquant-allow-lossy true --pngquant-quality 0..$quality -r ."

  echo "Done optimizing dir $dir"
}

I have an issue

ubuntu@ip-172-31-11-50:~/backend$ ls -al /tmp/asdfasdf
total 12
drwxrwxr-x  2 ubuntu ubuntu 4096 Jun 25 11:14 .
drwxrwxrwt 11 root   root   4096 Jun 25 11:01 ..
-rw-rw-r--  1 ubuntu ubuntu  130 May 15 19:32 CaravanersFlyHonduranFlagOnUSborder-wpFPapr30-18.png
ubuntu@ip-172-31-11-50:~/backend$ optimize_images 85 /tmp/asdfasdf
Optimizing dir /tmp/asdfasdf, quality - 85
Successfully installed fspath-3.1.0
Successfully installed image_size-1.5.0
Successfully installed exifr-1.3.4
Successfully installed progress-3.4.0
Successfully installed in_threads-1.5.0
Rails image assets optimization is extracted into image_optim_rails gem
You can safely remove `config.assets.image_optim = false` if you are not going to use that gem
Successfully installed image_optim-0.26.1
Successfully installed image_optim_pack-0.5.1-x86_64-linux
7 gems installed
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   98k  100   98k    0     0  71242      0  0:00:01  0:00:01 --:--:-- 71261
image_optim v0.26.1
config:
  threads: false
  verbose: true
  allow_lossy: true
  jpegoptim:
    allow_lossy: true
    max_quality: 85
  pngquant:
    allow_lossy: true
    quality: !ruby/range
      begin: 0
      end: 85
      excl: false
nice: 10
threads: 1
pack: true
skip_missing_workers: true
allow_lossy: true
cache_dir:
cache_worker_digests: false
image_optim_pack: all bins from /usr/local/bundle/gems/image_optim_pack-0.5.1-x86_64-linux/vendor/linux-x86_64 worked
Resolved pngcrush 1.8.13 at /usr/local/bundle/gems/image_optim_pack-0.5.1-x86_64-linux/vendor/linux-x86_64/pngcrush
Resolved pngout 20150319 at /usr/bin/pngout
Resolved advpng 2.1 at /usr/local/bundle/gems/image_optim_pack-0.5.1-x86_64-linux/vendor/linux-x86_64/advpng
Resolved optipng 0.7.7 at /usr/local/bundle/gems/image_optim_pack-0.5.1-x86_64-linux/vendor/linux-x86_64/optipng
Resolved pngquant 2.12.0 at /usr/local/bundle/gems/image_optim_pack-0.5.1-x86_64-linux/vendor/linux-x86_64/pngquant
Resolved jhead 3.00 at /usr/local/bundle/gems/image_optim_pack-0.5.1-x86_64-linux/vendor/linux-x86_64/jhead
Resolved jpegtran 9c at /usr/local/bundle/gems/image_optim_pack-0.5.1-x86_64-linux/vendor/linux-x86_64/jpegtran
Resolved jpegoptim 1.4.6 at /usr/local/bundle/gems/image_optim_pack-0.5.1-x86_64-linux/vendor/linux-x86_64/jpegoptim
Resolved jpeg-recompress 2.1.1 at /usr/local/bundle/gems/image_optim_pack-0.5.1-x86_64-linux/vendor/linux-x86_64/jpeg-recompress
Resolved jpegrescan 1a762f62 at /usr/local/bundle/gems/image_optim-0.26.1/vendor/jpegrescan
Resolved gifsicle 1.91 at /usr/local/bundle/gems/image_optim_pack-0.5.1-x86_64-linux/vendor/linux-x86_64/gifsicle
svgo worker: `svgo` not found; please provide proper binary or disable this worker (--no-svgo argument or `:svgo => false` through options)
Workers by format:
jpeg:
  jhead:
  jpegrecompress:
    allow_lossy: true
    quality: 3
  jpegoptim:
    allow_lossy: true
    strip: ["all"]
    max_quality: 85
  jpegtran:
    copy_chunks: false
    progressive: true
    jpegrescan: true
png:
  pngcrush:
    chunks: ["alla"]
    fix: false
    brute: false
    blacken: true
  optipng:
    level: 6
    interlace: false
    strip: true
  pngquant:
    allow_lossy: true
    max_colors: 256
    quality: 0..85
    speed: 3
  pngout:
    copy_chunks: false
    strategy: 0
  advpng:
    level: 4
gif:
  gifsicle:
    interlace: false
    level: 3
    careful: false
  gifsicle:
    interlace: true
    level: 3
    careful: false
optimizing: ......
✓ 0.00178151s pngcrush -reduce -cc -q -rem alla -blacken -- ./CaravanersFlyHonduranFlagOnUSborder-wpFPapr30-18.png /tmp/CaravanersFlyHonduranFlagOnUSborder-wpFPapr30-1820180625-1-npszes.png
✗ 0.00169076s optipng -strip all -i0 -o 6 -quiet -- /tmp/CaravanersFlyHonduranFlagOnUSborder-wpFPapr30-1820180625-1-npszes.png
✗ 0.00164564s pngquant --quality\=0-85 --speed\=3 --output\=/tmp/CaravanersFlyHonduranFlagOnUSborder-wpFPapr30-1820180625-1-npszes.png --force 256 -- ./CaravanersFlyHonduranFlagOnUSborder-wpFPapr30-18.png
optimizing: ...... (elapsed: 0s)
/usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim/cmd.rb:45:in `check_status!': SIGSEGV (SignalException)
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim/cmd.rb:13:in `run'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim/worker.rb:155:in `run_command'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim/worker.rb:127:in `execute'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim/worker/pngout.rb:34:in `optimize'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim.rb:112:in `block (4 levels) in optimize_image'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim/handler.rb:41:in `process'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim.rb:111:in `block (3 levels) in optimize_image'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim.rb:110:in `each'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim.rb:110:in `block (2 levels) in optimize_image'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim/handler.rb:25:in `for'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim.rb:109:in `block in optimize_image'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim/cache.rb:24:in `fetch'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim.rb:108:in `optimize_image'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim.rb:126:in `optimize_image!'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim.rb:246:in `block in run_method_for'
        from /usr/local/bundle/gems/progress-3.4.0/lib/progress/with_progress.rb:101:in `block (3 levels) in run_with_length'
        from /usr/local/bundle/gems/progress-3.4.0/lib/progress.rb:80:in `step'
        from /usr/local/bundle/gems/progress-3.4.0/lib/progress/class_methods.rb:25:in `step'
        from /usr/local/bundle/gems/progress-3.4.0/lib/progress/with_progress.rb:100:in `block (2 levels) in run_with_length'
        from /usr/local/bundle/gems/progress-3.4.0/lib/progress/with_progress.rb:99:in `map'
        from /usr/local/bundle/gems/progress-3.4.0/lib/progress/with_progress.rb:99:in `block in run_with_length'
        from /usr/local/bundle/gems/progress-3.4.0/lib/progress/class_methods.rb:16:in `start'
        from /usr/local/bundle/gems/progress-3.4.0/lib/progress/with_progress.rb:98:in `run_with_length'
        from /usr/local/bundle/gems/progress-3.4.0/lib/progress/with_progress.rb:68:in `run'
        from /usr/local/bundle/gems/progress-3.4.0/lib/progress/with_progress.rb:45:in `method_missing'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim.rb:245:in `run_method_for'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim.rb:159:in `optimize_images!'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim/runner.rb:76:in `optimize_images!'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim/runner.rb:62:in `run!'
        from /usr/local/bundle/gems/image_optim-0.26.1/bin/image_optim:20:in `<top (required)>'
        from /usr/local/bundle/bin/image_optim:23:in `load'
        from /usr/local/bundle/bin/image_optim:23:in `<main>'
Done optimizing dir /tmp/asdfasdf

With $stderr << "#{cmd_args.shelljoin}\n" as you asked

$   docker run -it --rm \
>     -v /tmp/asdfasdf:/usr/src/app \
>     -w /usr/src/app ruby:2.4-stretch bash
root@cdef5ac655e0:/usr/src/app#
root@cdef5ac655e0:/usr/src/app# gem install image_optim image_optim_pack
root@cdef5ac655e0:/usr/src/app# (curl -L "http://static.jonof.id.au/dl/kenutils/pngout-20150319-linux.tar.gz" | tar -xz -C /usr/bin --strip-components 2 --wildcards "*/x86_64/pngout")
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   98k  100   98k    0     0  71483      0  0:00:01  0:00:01 --:--:-- 71463

root@cdef5ac655e0:/usr/src/app# gem which image_optim
/usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim.rb

root@cdef5ac655e0:/usr/src/app# apt-get update
root@cdef5ac655e0:/usr/src/app# apt-get install vim
root@cdef5ac655e0:/usr/src/app# vim /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim/worker.rb

      $stderr << "\n\nHERERERERE!!!!\n"
      $stderr << "#{cmd_args.shelljoin}\n\n\n"

root@cdef5ac655e0:/usr/src/app# export quality="85"
root@cdef5ac655e0:/usr/src/app# image_optim --no-threads --verbose --allow-lossy --jpegoptim-allow-lossy true --jpegoptim-max-quality $quality --pngquant-allow-lossy true --pngquant-quality 0..$quality -r .
image_optim v0.26.1
config:
  threads: false
  verbose: true
  allow_lossy: true
  jpegoptim:
    allow_lossy: true
    max_quality: 85
  pngquant:
    allow_lossy: true
    quality: !ruby/range
      begin: 0
      end: 85
      excl: false
nice: 10
threads: 1
pack: true
skip_missing_workers: true
allow_lossy: true
cache_dir:
cache_worker_digests: false
image_optim_pack: all bins from /usr/local/bundle/gems/image_optim_pack-0.5.1-x86_64-linux/vendor/linux-x86_64 worked
Resolved pngcrush 1.8.13 at /usr/local/bundle/gems/image_optim_pack-0.5.1-x86_64-linux/vendor/linux-x86_64/pngcrush
Resolved pngout 20150319 at /usr/bin/pngout
Resolved advpng 2.1 at /usr/local/bundle/gems/image_optim_pack-0.5.1-x86_64-linux/vendor/linux-x86_64/advpng
Resolved optipng 0.7.7 at /usr/local/bundle/gems/image_optim_pack-0.5.1-x86_64-linux/vendor/linux-x86_64/optipng
Resolved pngquant 2.12.0 at /usr/local/bundle/gems/image_optim_pack-0.5.1-x86_64-linux/vendor/linux-x86_64/pngquant
Resolved jhead 3.00 at /usr/local/bundle/gems/image_optim_pack-0.5.1-x86_64-linux/vendor/linux-x86_64/jhead
Resolved jpegtran 9c at /usr/local/bundle/gems/image_optim_pack-0.5.1-x86_64-linux/vendor/linux-x86_64/jpegtran
Resolved jpegoptim 1.4.6 at /usr/local/bundle/gems/image_optim_pack-0.5.1-x86_64-linux/vendor/linux-x86_64/jpegoptim
Resolved jpeg-recompress 2.1.1 at /usr/local/bundle/gems/image_optim_pack-0.5.1-x86_64-linux/vendor/linux-x86_64/jpeg-recompress
Resolved jpegrescan 1a762f62 at /usr/local/bundle/gems/image_optim-0.26.1/vendor/jpegrescan
Resolved gifsicle 1.91 at /usr/local/bundle/gems/image_optim_pack-0.5.1-x86_64-linux/vendor/linux-x86_64/gifsicle
svgo worker: `svgo` not found; please provide proper binary or disable this worker (--no-svgo argument or `:svgo => false` through options)
Workers by format:
jpeg:
  jhead:
  jpegrecompress:
    allow_lossy: true
    quality: 3
  jpegoptim:
    allow_lossy: true
    strip: ["all"]
    max_quality: 85
  jpegtran:
    copy_chunks: false
    progressive: true
    jpegrescan: true
png:
  pngcrush:
    chunks: ["alla"]
    fix: false
    brute: false
    blacken: true
  optipng:
    level: 6
    interlace: false
    strip: true
  pngquant:
    allow_lossy: true
    max_colors: 256
    quality: 0..85
    speed: 3
  pngout:
    copy_chunks: false
    strategy: 0
  advpng:
    level: 4
gif:
  gifsicle:
    interlace: false
    level: 3
    careful: false
  gifsicle:
    interlace: true
    level: 3
    careful: false
optimizing: ......

HERERERERE!!!!
pngcrush -reduce -cc -q -rem alla -blacken -- ./CaravanersFlyHonduranFlagOnUSborder-wpFPapr30-18.png /tmp/CaravanersFlyHonduranFlagOnUSborder-wpFPapr30-1820180625-459-1mv1dpr.png

✓ 0.001675792s pngcrush -reduce -cc -q -rem alla -blacken -- ./CaravanersFlyHonduranFlagOnUSborder-wpFPapr30-18.png /tmp/CaravanersFlyHonduranFlagOnUSborder-wpFPapr30-1820180625-459-1mv1dpr.png

HERERERERE!!!!
optipng -strip all -i0 -o 6 -quiet -- /tmp/CaravanersFlyHonduranFlagOnUSborder-wpFPapr30-1820180625-459-1mv1dpr.png

✗ 0.001588589s optipng -strip all -i0 -o 6 -quiet -- /tmp/CaravanersFlyHonduranFlagOnUSborder-wpFPapr30-1820180625-459-1mv1dpr.png

HERERERERE!!!!
pngquant --quality\=0-85 --speed\=3 --output\=/tmp/CaravanersFlyHonduranFlagOnUSborder-wpFPapr30-1820180625-459-1mv1dpr.png --force 256 -- ./CaravanersFlyHonduranFlagOnUSborder-wpFPapr30-18.png

✗ 0.001635957s pngquant --quality\=0-85 --speed\=3 --output\=/tmp/CaravanersFlyHonduranFlagOnUSborder-wpFPapr30-1820180625-459-1mv1dpr.png --force 256 -- ./CaravanersFlyHonduranFlagOnUSborder-wpFPapr30-18.png

HERERERERE!!!!
pngout -k0 -s0 -q -y ./CaravanersFlyHonduranFlagOnUSborder-wpFPapr30-18.png /tmp/CaravanersFlyHonduranFlagOnUSborder-wpFPapr30-1820180625-459-1mv1dpr.png

optimizing: ...... (elapsed: 0s)
/usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim/cmd.rb:45:in `check_status!': SIGSEGV (SignalException)
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim/cmd.rb:13:in `run'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim/worker.rb:157:in `run_command'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim/worker.rb:129:in `execute'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim/worker/pngout.rb:34:in `optimize'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim.rb:112:in `block (4 levels) in optimize_image'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim/handler.rb:41:in `process'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim.rb:111:in `block (3 levels) in optimize_image'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim.rb:110:in `each'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim.rb:110:in `block (2 levels) in optimize_image'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim/handler.rb:25:in `for'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim.rb:109:in `block in optimize_image'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim/cache.rb:24:in `fetch'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim.rb:108:in `optimize_image'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim.rb:126:in `optimize_image!'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim.rb:246:in `block in run_method_for'
        from /usr/local/bundle/gems/progress-3.4.0/lib/progress/with_progress.rb:101:in `block (3 levels) in run_with_length'
        from /usr/local/bundle/gems/progress-3.4.0/lib/progress.rb:80:in `step'
        from /usr/local/bundle/gems/progress-3.4.0/lib/progress/class_methods.rb:25:in `step'
        from /usr/local/bundle/gems/progress-3.4.0/lib/progress/with_progress.rb:100:in `block (2 levels) in run_with_length'
        from /usr/local/bundle/gems/progress-3.4.0/lib/progress/with_progress.rb:99:in `map'
        from /usr/local/bundle/gems/progress-3.4.0/lib/progress/with_progress.rb:99:in `block in run_with_length'
        from /usr/local/bundle/gems/progress-3.4.0/lib/progress/class_methods.rb:16:in `start'
        from /usr/local/bundle/gems/progress-3.4.0/lib/progress/with_progress.rb:98:in `run_with_length'
        from /usr/local/bundle/gems/progress-3.4.0/lib/progress/with_progress.rb:68:in `run'
        from /usr/local/bundle/gems/progress-3.4.0/lib/progress/with_progress.rb:45:in `method_missing'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim.rb:245:in `run_method_for'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim.rb:159:in `optimize_images!'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim/runner.rb:76:in `optimize_images!'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim/runner.rb:62:in `run!'
        from /usr/local/bundle/gems/image_optim-0.26.1/bin/image_optim:20:in `<top (required)>'
        from /usr/local/bundle/bin/image_optim:23:in `load'
        from /usr/local/bundle/bin/image_optim:23:in `<main>'
Segmentation fault (core dumped)
root@cdef5ac655e0:/usr/src/app#
srghma commented 6 years ago

the file that is causing the issue

https://mega.nz/#!g90EVAYL!Hwl_VwD4IM-Wud3DMUkp-tEtNtDIW5JXMLw6bEvFBbc

srghma commented 6 years ago 
✗ 0.572604755s pngquant --quality\=0-85 --speed\=3 --output\=/tmp/CaravanersFlyHonduranFlagOnUSborder-wpFPapr30-1820180625-384-199999l.png --force 256 -- ./CaravanersFlyHonduranFlagOnUSborder-wpFPapr30-18.png

From: /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim/cmd.rb @ line 34 ImageOptim::Cmd.check_status!:

    31:       def check_status!
    32:         status = $CHILD_STATUS
    33:
 => 34:         require 'pry'; ::Kernel.binding.pry;
    35:         return unless status.signaled?
    36:
    37:         # jruby incorrectly returns true for `signaled?` if process exits with
    38:         # non zero status. For following code
    39:         #
    40:         #     `sh -c 'exit 66'`
    41:         #     p [$?.signaled?, $?.exitstatus, $?.termsig]
    42:         #
    43:         # jruby outputs `[true, 66, 66]` instead of expected `[false, 66, nil]`
    44:         return if defined?(JRUBY_VERSION) && status.exitstatus == status.termsig
    45:
    46:         fail SignalException, status.termsig
    47:       end

[1] pry(ImageOptim::Cmd)> status
=> #<Process::Status: pid 431 SIGSEGV (signal 11)>
[2] pry(ImageOptim::Cmd)> status.signaled?
=> true
[3] pry(ImageOptim::Cmd)> status.signaled?
=> true
[4] pry(ImageOptim::Cmd)> [$?.signaled?, $?.exitstatus, $?.termsig]
=> [true, nil, 11]
optimizing: ......
optimizing: ...... (elapsed: 4.9m)
/usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim/cmd.rb:46:in `check_status!': SIGSEGV (SignalException)
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim/cmd.rb:13:in `run'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim/worker.rb:155:in `run_command'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim/worker.rb:127:in `execute'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim/worker/pngout.rb:34:in `optimize'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim.rb:112:in `block (4 levels) in optimize_image'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim/handler.rb:41:in `process'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim.rb:111:in `block (3 levels) in optimize_image'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim.rb:110:in `each'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim.rb:110:in `block (2 levels) in optimize_image'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim/handler.rb:25:in `for'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim.rb:109:in `block in optimize_image'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim/cache.rb:24:in `fetch'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim.rb:108:in `optimize_image'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim.rb:126:in `optimize_image!'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim.rb:246:in `block in run_method_for'
        from /usr/local/bundle/gems/progress-3.4.0/lib/progress/with_progress.rb:101:in `block (3 levels) in run_with_length'
        from /usr/local/bundle/gems/progress-3.4.0/lib/progress.rb:80:in `step'
        from /usr/local/bundle/gems/progress-3.4.0/lib/progress/class_methods.rb:25:in `step'
        from /usr/local/bundle/gems/progress-3.4.0/lib/progress/with_progress.rb:100:in `block (2 levels) in run_with_length'
        from /usr/local/bundle/gems/progress-3.4.0/lib/progress/with_progress.rb:99:in `map'
        from /usr/local/bundle/gems/progress-3.4.0/lib/progress/with_progress.rb:99:in `block in run_with_length'
        from /usr/local/bundle/gems/progress-3.4.0/lib/progress/class_methods.rb:16:in `start'
        from /usr/local/bundle/gems/progress-3.4.0/lib/progress/with_progress.rb:98:in `run_with_length'
        from /usr/local/bundle/gems/progress-3.4.0/lib/progress/with_progress.rb:68:in `run'
        from /usr/local/bundle/gems/progress-3.4.0/lib/progress/with_progress.rb:45:in `method_missing'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim.rb:245:in `run_method_for'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim.rb:159:in `optimize_images!'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim/runner.rb:76:in `optimize_images!'
        from /usr/local/bundle/gems/image_optim-0.26.1/lib/image_optim/runner.rb:62:in `run!'
        from /usr/local/bundle/gems/image_optim-0.26.1/bin/image_optim:20:in `<top (required)>'
        from /usr/local/bundle/bin/image_optim:23:in `load'
        from /usr/local/bundle/bin/image_optim:23:in `<main>'
Segmentation fault
root@51d4a8673af9:/usr/src/app#

signalling means that someone requested program to exist, I'm not sure why pngquant stopps with this signal

srghma commented 6 years ago

I decided to just ignore this error https://github.com/srghma/image_optim/commit/66b8a2bd904cc7bea8544230ba9806e9717885c7

it works, though Ctrl-c is ignored )))))

toy commented 6 years ago

SIGSEGV is "Segmentation fault", maybe it is better to not throw a SignalException in cmd.rb. Docker should make it easy to reproduce the error, I'll have a look.

srghma commented 6 years ago

Ok, yes, so basically what we want here is to ignore "Segmentation fault" of child and not to ignore ctrl-c

toy commented 6 years ago

Reupload to github storage: pngout-killer

toy commented 6 years ago

Have a look at branch pngout-ignore-segmentation-fault.

srghma commented 6 years ago

@toy, hi, it works

optimize_images () {
  quality="$1"
  dir="$2"

  echo "Optimizing dir $dir, quality - $quality"

  docker run --rm \
    -v $dir:/usr/src/app \
    -w /usr/src/app ruby:2.4-stretch bash -c \
    "git clone https://github.com/toy/image_optim /image_optim && \
    (\
      cd /image_optim && \
      git checkout pngout-ignore-segmentation-fault && \
      bundle install && \
      gem build image_optim.gemspec && \
      gem install image_optim-0.26.1.gem \
    ) && \
    gem install image_optim_pack && \
    (curl -L \"http://static.jonof.id.au/dl/kenutils/pngout-20150319-linux.tar.gz\" | tar -xz -C /usr/bin --strip-components 2 --wildcards \"*/x86_64/pngout\") && \
    image_optim --no-threads --verbose --allow-lossy --jpegoptim-allow-lossy true --jpegoptim-max-quality $quality --pngquant-allow-lossy true --pngquant-quality 0..$quality -r ."

  echo "Done optimizing dir $dir"
}
 ~/Downloads/dir  optimize_images 85 $(realpath .)
Optimizing dir /home/srghma/Downloads/dir, quality - 85
Cloning into '/image_optim'...
Switched to a new branch 'pngout-ignore-segmentation-fault'
Branch pngout-ignore-segmentation-fault set up to track remote branch pngout-ignore-segmentation-fault from origin.
Fetching gem metadata from https://rubygems.org/.......
Resolving dependencies...
Fetching ast 2.4.0
Installing ast 2.4.0
Using bundler 1.16.1
Fetching diff-lcs 1.3
Installing diff-lcs 1.3
Fetching exifr 1.3.4
Installing exifr 1.3.4
Fetching fspath 3.1.0
Installing fspath 3.1.0
Fetching image_size 2.0.0
Installing image_size 2.0.0
Fetching in_threads 1.5.0
Installing in_threads 1.5.0
Fetching progress 3.4.0
Installing progress 3.4.0
Using image_optim 0.26.1 from source at `.`
Fetching image_optim_pack 0.5.1 (x86_64-linux)
Installing image_optim_pack 0.5.1 (x86_64-linux)
Fetching jaro_winkler 1.5.1
Installing jaro_winkler 1.5.1 with native extensions
Fetching parallel 1.12.1
Installing parallel 1.12.1
Fetching parser 2.5.1.2
Installing parser 2.5.1.2
Fetching powerpack 0.1.2
Installing powerpack 0.1.2
Fetching rainbow 3.0.0
Installing rainbow 3.0.0
Fetching rspec-support 3.7.1
Installing rspec-support 3.7.1
Fetching rspec-core 3.7.1
Installing rspec-core 3.7.1
Fetching rspec-expectations 3.7.0
Installing rspec-expectations 3.7.0
Fetching rspec-mocks 3.7.0
Installing rspec-mocks 3.7.0
Fetching rspec 3.7.0
Installing rspec 3.7.0
Fetching ruby-progressbar 1.9.0
Installing ruby-progressbar 1.9.0
Fetching unicode-display_width 1.4.0
Installing unicode-display_width 1.4.0
Fetching rubocop 0.58.2
Installing rubocop 0.58.2
Fetching travis_check_rubies 0.2.5
Installing travis_check_rubies 0.2.5
Bundle complete! 5 Gemfile dependencies, 24 gems now installed.
Bundled gems are installed into `/usr/local/bundle`
  Successfully built RubyGem
  Name: image_optim
  Version: 0.26.1
  File: image_optim-0.26.1.gem
Rails image assets optimization is extracted into image_optim_rails gem
You can safely remove `config.assets.image_optim = false` if you are not going to use that gem
Successfully installed image_optim-0.26.1
1 gem installed
Successfully installed image_optim_pack-0.5.1-x86_64-linux
1 gem installed
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   98k  100   98k    0     0  25237      0  0:00:03  0:00:03 --:--:-- 25237
image_optim v0.26.1
config:
  threads: false
  verbose: true
  allow_lossy: true
  jpegoptim:
    allow_lossy: true
    max_quality: 85
  pngquant:
    allow_lossy: true
    quality: !ruby/range
      begin: 0
      end: 85
      excl: false
nice: 10
threads: 1
pack: true
skip_missing_workers: true
allow_lossy: true
cache_dir:
cache_worker_digests: false
image_optim_pack: all bins from /usr/local/bundle/gems/image_optim_pack-0.5.1-x86_64-linux/vendor/linux-x86_64 worked
PATH: /usr/local/bundle/gems/image_optim_pack-0.5.1-x86_64-linux/vendor/linux-x86_64:/usr/local/bundle/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/bundle/gems/image_optim-0.26.1/vendor
Resolved pngcrush 1.8.13 at /usr/local/bundle/gems/image_optim_pack-0.5.1-x86_64-linux/vendor/linux-x86_64/pngcrush
Resolved pngout 20150319 at /usr/bin/pngout
Resolved advpng 2.1 at /usr/local/bundle/gems/image_optim_pack-0.5.1-x86_64-linux/vendor/linux-x86_64/advpng
Resolved optipng 0.7.7 at /usr/local/bundle/gems/image_optim_pack-0.5.1-x86_64-linux/vendor/linux-x86_64/optipng
Resolved pngquant 2.12.0 at /usr/local/bundle/gems/image_optim_pack-0.5.1-x86_64-linux/vendor/linux-x86_64/pngquant
Resolved jhead 3.00 at /usr/local/bundle/gems/image_optim_pack-0.5.1-x86_64-linux/vendor/linux-x86_64/jhead
Resolved jpegtran 9c at /usr/local/bundle/gems/image_optim_pack-0.5.1-x86_64-linux/vendor/linux-x86_64/jpegtran
Resolved jpegoptim 1.4.6 at /usr/local/bundle/gems/image_optim_pack-0.5.1-x86_64-linux/vendor/linux-x86_64/jpegoptim
Resolved jpeg-recompress 2.1.1 at /usr/local/bundle/gems/image_optim_pack-0.5.1-x86_64-linux/vendor/linux-x86_64/jpeg-recompress
Resolved jpegrescan 1a762f62 at /usr/local/bundle/gems/image_optim-0.26.1/vendor/jpegrescan
Resolved gifsicle 1.91 at /usr/local/bundle/gems/image_optim_pack-0.5.1-x86_64-linux/vendor/linux-x86_64/gifsicle
svgo worker: `svgo` not found; please provide proper binary or disable this worker (--no-svgo argument or `:svgo => false` through options)
Workers by format:
jpeg:
  jhead:
  jpegrecompress:
    allow_lossy: true
    quality: 3
  jpegoptim:
    allow_lossy: true
    strip: ["all"]
    max_quality: 85
  jpegtran:
    copy_chunks: false
    progressive: true
    jpegrescan: true
png:
  pngcrush:
    chunks: ["alla"]
    fix: false
    brute: false
    blacken: true
  optipng:
    level: 6
    interlace: false
    strip: true
  pngquant:
    allow_lossy: true
    max_colors: 256
    quality: 0..85
    speed: 3
  pngout:
    copy_chunks: false
    strategy: 0
  advpng:
    level: 4
gif:
  gifsicle:
    interlace: false
    level: 3
    careful: false
  gifsicle:
    interlace: true
    level: 3
    careful: false
optimizing: ......
✓ 0.001887419s pngcrush -reduce -cc -q -rem alla -blacken -- ./CaravanersFlyHonduranFlagOnUSborder-wpFPapr30-18.png /tmp/CaravanersFlyHonduranFlagOnUSborder-wpFPapr30-1820180729-1-1xxlliu.png
✗ 0.001704362s optipng -strip all -i0 -o 6 -quiet -- /tmp/CaravanersFlyHonduranFlagOnUSborder-wpFPapr30-1820180729-1-1xxlliu.png
✗ 0.001930648s pngquant --quality\=0-85 --speed\=3 --output\=/tmp/CaravanersFlyHonduranFlagOnUSborder-wpFPapr30-1820180729-1-1xxlliu.png --force 256 -- ./CaravanersFlyHonduranFlagOnUSborder-wpFPapr30-18.png
pngout caused Segmentation fault for ./CaravanersFlyHonduranFlagOnUSborder-wpFPapr30-18.png
✗ 0.00901605s advpng --recompress -4 --quiet -- /tmp/CaravanersFlyHonduranFlagOnUSborder-wpFPapr30-1820180729-1-1xxlliu.png
optimizing: 100.0%
optimizing: 100.0% (elapsed: 0s)
------         ./CaravanersFlyHonduranFlagOnUSborder-wpFPapr30-18.png
Total: ------
Done optimizing dir /home/srghma/Downloads/dir
srghma commented 6 years ago

tnx @toy