imagemin silently drops files

[andrenarchy]

With the configuration

    .pipe(imagemin([
      imagemin.gifsicle({interlaced: true}),
      imagemin.jpegtran({progressive: true}),
      imagemin.optipng(),
      imagemin.svgo([{removeViewBox: false}, {minifyStyles: false}])
    ], {verbose: true}))

only ~30 of my ~100 files are optimized. The rest is silently dropped, i.e., they are just missing in the output directory.

[kevva]

What type of images are getting lost? There are some issues open about JPG images not working.

[andrenarchy]

SVG, PNG and JPG.

[kevva]

Only some of them, or all?

[andrenarchy]

Only some. You can reproduce the behavior with the following commands

mkdir /tmp/imagemin-test && cd /tmp/imagemin-test
git clone https://github.com/paperhive/paperhive-frontend-static.git static
npm init -y
npm install gulp gulp-imagemin

Running gulp static with the following gulpfile.js

var gulp = require('gulp');
var imagemin = require('gulp-imagemin');

gulp.task('static', function() {
  return gulp.src('static/**/*')
    .pipe(imagemin([
      imagemin.gifsicle({interlaced: true}),
      imagemin.jpegtran({progressive: true}),
      imagemin.optipng(),
      imagemin.svgo([{removeViewBox: false}, {minifyStyles: false}])
    ], {verbose: true}))
    .pipe(gulp.dest('build-static'));
});

Then compare the content of static/img with build-static/img:

$ find static/img -type f | wc -l
67
$ find build-static/img -type f | wc -l
3

[kevva]

Hmm, I can reproduce, but I have no idea where it happens. I don't get all the files even if I log file here https://github.com/sindresorhus/gulp-imagemin/blob/master/index.js#L32.

[andrenarchy]

Maybe an updated dependency introduced the issue? It definitely still works with 2.4.0.

[drose-ox]

I have the same problem. If I comment out the line pipe(imagemin(...)) in my gulpfile I get all images in my output directory. If I leave the line: "gulp-imagemin: Minified 0 images"

No images present.

[s100]

I think there are problems with the way errors in plugins are being handled. In my case I discovered there was a syntax error in one of my SVG files, but the appropriate stack trace only became visible when I manually reduced my collection of plugins to just svgo by itself. Adding any earlier or later plugins to the array being passed to imagemin caused the error to silently disappear.

A problem with gifsicle was just swallowed up entirely.

[kirillgroshkov]

+1. I reproduce the problem with imagemin-jpeg-recompress on windows - drops files (actually, more than 95% of the files got dropped) when I use method 'smallfry'. Using other methods work (no drops).

[kirillgroshkov]

Debugged a bit deeper today, was able to identify 3 issues, bottom up:

1. imagemin-jpeg-recompress cannot process some (~1%) of my files, throws an error (only on method 'smallfry'). I will post my samples later. Same files are processed without a problem when other methods are used ('mpe', 'ssim')
2. imagemin.buffer is not catching these errors and failing silently
3. When gulp-imagemin catches an error - it calls 'cb(new gutil.PluginError ...' that is passed to 'through', and later it's causing gulp-imagemin to sliently stop, without any messages. It stops not immediately on the erroneous image, but 10-20 images later (due to concurrency, I guess).

Altogether it "silently fails" without giving any indication of a problem.

[aarongustafson]

Yup, running into this too. Using

gulp.src( source + '/**/*.{jpg,png,svg,gif}')
    .pipe(gulp.dest(destination))
    .pipe(filelog('Preparing'))
    .pipe(imagemin()) // Optimize
    .pipe(filelog('Optimized'))
    ;

I get a log of files going in and files coming out of imagemin:

[11:25:57] [Preparing] [1] [/deploy/i/hero.png]
[11:25:57] [Preparing] [2] [/deploy/i/offline.png]
[11:25:57] [Preparing] [3] [/deploy/i/hero.svg]
[11:25:57] [Preparing] [4] [/deploy/i/c/aea.png]
[11:25:57] [Preparing] [5] [/deploy/i/c/edge.png]
[11:25:57] [Preparing] [6] [/deploy/i/c/o.png]
[11:25:57] [Preparing] [7] [/deploy/i/c/star.png]
[11:25:57] [Preparing] [8] [/deploy/i/c/aea.svg]
[11:25:57] [Preparing] [9] [/deploy/i/c/edge.svg]
[11:25:57] [Preparing] [10] [/deploy/i/c/o.svg]
[11:25:57] [Preparing] [11] [/deploy/i/c/star.svg]
[11:25:57] [Preparing] [12] [/deploy/i/j/a.jpg]
[11:25:57] [Preparing] [13] [/deploy/i/j/h.jpg]
[11:25:57] [Preparing] [14] [/deploy/i/j/j.jpg]
[11:25:57] [Preparing] [15] [/deploy/i/j/l.jpg]
[11:25:57] [Preparing] [16] [/deploy/i/j/m.jpg]
[11:25:57] [Preparing] [17] [/deploy/i/j/r.jpg]
[11:25:57] [Preparing] [18] [/deploy/i/j/s.jpg]
[11:25:57] [Preparing] [19] [/deploy/i/j/offline.png]
[11:25:58] gulp-imagemin: ✔ hero.png (saved 25.22 kB - 36.2%)
[11:25:58] [Optimized] [1] [/deploy/i/hero.png]
[11:25:58] gulp-imagemin: ✔ offline.png (already optimized)
[11:25:58] [Optimized] [2] [/deploy/i/offline.png]

As you can see, 19 went in, only 2 came out. And this is with verbose logging on.

At first I thought the subdirectories were being lost, but even hero.svg in the root folder is skipped (and yes, it is a valid SVG).

gulp-imagemin: 3.0.1 OS: OS X 10.10.5 Node: 6.3.0

[aarongustafson]

Ok, it turns out a bug in one of my SVGs was causing the silent failure. It would be helpful to expose that, especially when we aren’t always the source of our SVGs :-)

[shinnn]

@aarongustafson Can you provide a SVG that causes the silent failure? I created a corrupt SVG (<svg><</svg>), tried to minify it with gulp-imagemin, and it showed the error message actually.

[11:03:02] Using gulpfile ~/github/MyRepositories/shinnn.github.io/gulpfile.js
[11:03:02] Starting 'images'...
Unhandled rejection Error: Error in parsing SVG: Unencoded <
Line: 0
Column: 7
Char: <
    at svgo.optimize.res (/Users/Shinnosuke/github/MyRepositories/shinnn.github.io/node_modules/imagemin-svgo/index.js:21:12)
    at optimizeOnceCallback (/Users/Shinnosuke/github/MyRepositories/shinnn.github.io/node_modules/svgo/lib/svgo.js:36:17)
    at /Users/Shinnosuke/github/MyRepositories/shinnn.github.io/node_modules/svgo/lib/svgo.js:59:13
    at module.exports (/Users/Shinnosuke/github/MyRepositories/shinnn.github.io/node_modules/svgo/lib/svgo/svg2js.js:166:9)
    at SVGO._optimizeOnce (/Users/Shinnosuke/github/MyRepositories/shinnn.github.io/node_modules/svgo/lib/svgo.js:56:5)
    at SVGO.optimize (/Users/Shinnosuke/github/MyRepositories/shinnn.github.io/node_modules/svgo/lib/svgo.js:49:11)
    at Promise (/Users/Shinnosuke/github/MyRepositories/shinnn.github.io/node_modules/imagemin-svgo/index.js:19:8)
    at buf (/Users/Shinnosuke/github/MyRepositories/shinnn.github.io/node_modules/imagemin-svgo/index.js:18:9)
    at /Users/Shinnosuke/github/MyRepositories/shinnn.github.io/node_modules/promise.pipe/index.js:25:18
    at process._tickCallback (internal/process/next_tick.js:103:7)

I want to reproduce the "silent failure".

[aarongustafson]

My best guess it it was a namespacing issue, the namespace for the xlink attribute was undefined. Here’s a reduction:

<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1332.7 858.1">
  <title>
    Won’t compress
  </title>
  <script xlink:href="https://cdnjs.cloudflare.com/ajax/libs/gsap/1.18.4/TweenMax.min.js" type="text/javascript"/>
</svg>

All SVGs after this one in the directory (alphabetically) are skipped too. I just tested it by naming the file z.svg and then a.svg and was able to reproduce the issue.

[shinnn]

Hmmmm... the corrupt SVG @aarongustafson provided doesn't cause the "silent failure" on my machine.

Unhandled rejection Error: Error in parsing SVG: Unbound namespace prefix: "xlink"
Line: 4
Column: 114
Char: >
    at svgo.optimize.res (/Users/Shinnosuke/github/MyRepositories/shinnn.github.io/node_modules/imagemin-svgo/index.js:21:12)
    at optimizeOnceCallback (/Users/Shinnosuke/github/MyRepositories/shinnn.github.io/node_modules/svgo/lib/svgo.js:36:17)
    at /Users/Shinnosuke/github/MyRepositories/shinnn.github.io/node_modules/svgo/lib/svgo.js:59:13
    at module.exports (/Users/Shinnosuke/github/MyRepositories/shinnn.github.io/node_modules/svgo/lib/svgo/svg2js.js:166:9)
    at SVGO._optimizeOnce (/Users/Shinnosuke/github/MyRepositories/shinnn.github.io/node_modules/svgo/lib/svgo.js:56:5)
    at SVGO.optimize (/Users/Shinnosuke/github/MyRepositories/shinnn.github.io/node_modules/svgo/lib/svgo.js:49:11)
    at Promise (/Users/Shinnosuke/github/MyRepositories/shinnn.github.io/node_modules/imagemin-svgo/index.js:19:8)
    at buf (/Users/Shinnosuke/github/MyRepositories/shinnn.github.io/node_modules/imagemin-svgo/index.js:18:9)
    at /Users/Shinnosuke/github/MyRepositories/shinnn.github.io/node_modules/promise.pipe/index.js:25:18
    at process._tickCallback (internal/process/next_tick.js:103:7)

@aarongustafson,

1. Do you return a gulp.src stream inside the task?
2. Can you reinstall whole npm packages and try again? It sometimes solves a problem. https://github.com/sindresorhus/gulp-imagemin/issues/173#issuecomment-230737023

rm -rf ./node_modules
npm cache clean
npm i

[aarongustafson]

@shinnn Still failed silently for me, even with --verbose on.

[15:25:04] Starting 'images'...
[15:25:04] Finished 'images' after 11 ms
[15:25:05] gulp-imagemin: ✔ hero.png (saved 25.22 kB - 36.2%)
[15:25:05] gulp-imagemin: ✔ offline.png (already optimized)

File was a.svg, there was also hero.svg in the folder.

[s100]

Those of you seeing a "noisy failure" rather than the reported silent failure, have you tried running a build with several images, only one of which (not the last one) is corrupted? This is the scenario I found difficulty with; the failure was swallowed and all the following images silently dropped.

[shinnn]

@aarongustafson @s100 Can you create an example repository? The repository includes package.json, gulpfile and some images, by which we can easily reproduce the problem with a simple npm start command.

[s100]

@shinnn Hope this helps: https://github.com/s100/gulp-imagemin-issue183.

[shinnn]

@s100 Thanks to your nicely straightforward example, I could soon find a solution and just submitted a PR. https://github.com/sindresorhus/gulp-imagemin/pull/199 💡

[aarongustafson]

Thanks @s100! I’m head-down working on a presentation for next week, so hadn’t had a chance to do it yet :-)