"Segmentation fault (core dumped)" when working on multiple directories concurrently

[ThePeach]

This is starting to bug me a bit too much, so I start praying the gods of gulp-responsive, hoping in a solution that can perhaps solve this issue.

I have a (probably) complex situation, and I've detailed it in a post on Stack Overflow.

I've now tried a much simpler approach (at least from a programmatic perspective), with a task that is defined as follows:

function buildPreviews () {
  const folders = ['/photo', '/vector', '/illustration', '/pixelart'];

  const tasks = folders.map((folder) => {
    const imagesFolder = path.join(paths.images.src, folder, `/preview/*.png`);

    return gulp.src(imagesFolder)
      .pipe(responsive({
        '*.png': [ {
          width: 120,
          rename: {
            dirname: 'preview/half'
          }
        } ]
      }, {
        progressive: true,
        withMetadata: false,
        errorOnEnlargement: false,
        errorOnUnusedConfig: false,
        errorOnUnusedImage: false
      }))
      .pipe(gulp.dest(path.join(paths.images.dest, folder)));
  });

  return merge(tasks);
}

When passing just one directory, everything works fine, as soon as I raise the bar, everything crashes randomly with the following error:

[snip]
(node:21002) DeprecationWarning: quality: use jpeg({ quality: ... }), webp({ quality: ... }) and/or tiff({ quality: ... }) instead
(node:21002) DeprecationWarning: progressive: use jpeg({ progressive: ... }) and/or png({ progressive: ... }) instead
(node:21002) DeprecationWarning: withoutChromaSubsampling: use jpeg({ chromaSubsampling: "4:4:4" }) instead
(node:21002) DeprecationWarning: compressionLevel: use png({ compressionLevel: ... }) instead
[22:53:19] gulp-responsive: 2004-09-27-HCS-logo.png -> preview/half/2004-09-27-HCS-logo.png
[22:53:19] gulp-responsive: Scan-130504-0003.png -> preview/half/Scan-130504-0003.png
[22:53:19] gulp-responsive: 2010-12-13-mother-cry.png -> preview/half/2010-12-13-mother-cry.png
[22:53:19] gulp-responsive: 2009-04-11-moai.png -> preview/half/2009-04-11-moai.png
[22:53:19] gulp-responsive: 2004-11-18-G-logo.png -> preview/half/2004-11-18-G-logo.png
[22:53:19] gulp-responsive: 2011-02-19-chameleon.png -> preview/half/2011-02-19-chameleon.png
[22:53:19] gulp-responsive: Scan-130527-0002.png -> preview/half/Scan-130527-0002.png
[22:53:20] gulp-responsive: 2009-05-08-angkor_wat.png -> preview/half/2009-05-08-angkor_wat.png
[22:53:20] gulp-responsive: 2011-02-football-player.png -> preview/half/2011-02-football-player.png
[22:53:20] gulp-responsive: 2004-11-23-larry_the_cow-head.png -> preview/half/2004-11-23-larry_the_cow-head.png
[22:53:20] gulp-responsive: Scan-140223-0008.png -> preview/half/Scan-140223-0008.png
[22:53:20] gulp-responsive: 2009-05-11-circleskate.png -> preview/half/2009-05-11-circleskate.png
[22:53:20] gulp-responsive: 2011-02-mario.png -> preview/half/2011-02-mario.png
[22:53:20] gulp-responsive: 2004-12-09-larry_the_cow-full.png -> preview/half/2004-12-09-larry_the_cow-full.png
[22:53:20] gulp-responsive: 2009-05-21-employee.png -> preview/half/2009-05-21-employee.png
[22:53:20] gulp-responsive: Scan-140223-0010.png -> preview/half/Scan-140223-0010.png
[22:53:20] gulp-responsive: 2011-03-15-the_plague.png -> preview/half/2011-03-15-the_plague.png
[22:53:20] gulp-responsive: 2005-05-25-gentoo-abducted_1280x800.png -> preview/half/2005-05-25-gentoo-abducted_1280x800.png
[22:53:20] gulp-responsive: 2009-06-14-hobo_heaven.png -> preview/half/2009-06-14-hobo_heaven.png
[22:53:20] gulp-responsive: Scan-140226-0005.png -> preview/half/Scan-140226-0005.png
[22:53:20] 'buildPreviews' errored after 683 ms
[22:53:20] Error: ENOENT: no such file or directory, open '/home/peach/public_html/portfolio.smartart.it/dist/images/illustration/preview/half/2011-03-15-the_plague.png'
[22:53:20] 'build' errored after 753 ms
[22:53:20] 'default' errored after 754 ms
[22:53:20] The following tasks did not complete: styles, metalsmith
[22:53:20] Did you forget to signal async completion?

(sharp:21002): GLib-CRITICAL **: g_hash_table_lookup: assertion 'hash_table != NULL' failed

(sharp:21002): GLib-CRITICAL **: g_hash_table_insert_internal: assertion 'hash_table != NULL' failed

(sharp:21002): GLib-CRITICAL **: g_hash_table_lookup: assertion 'hash_table != NULL' failed
Segmentation fault (core dumped)

The file the error is triggered from seems to vary.

So here I am, desperate :sob:, ignorant of what works behind the scenes and asking: am I doing something wrong?

I thought of writing a single config, but then I don't know how to allow gulp-responsive to write in the right directory...

[TheDancingCode]

I don't know if it fixes your problem, but you can certainly simplify your task as follows:

gulp.task('buildPreviews', function() {
  const folders = ['photo', 'vector', 'illustration', 'pixelart'];
  const glob = `${paths.images.src}/{${folders.join(',')}}/preview/*.png`

  return gulp.src(glob)
    .pipe(responsive(
        {
          '**/*.png': [
            {
              width: 120,
              rename: {
                prefix: 'half/'
              }
            }
          ]
        },
        {
          progressive: true,
          withMetadata: false,
          errorOnEnlargement: false,
          errorOnUnusedConfig: false,
          errorOnUnusedImage: false
        }
      )
    )
    .pipe(gulp.dest(paths.images.dest));
});

[ThePeach]

thanks @TheDancingCode, your code works without crashing, but unfortunately it doesn't adapt to my situation, as I would need to save the images in different subfolders: in your case I would get all the processed images in the same folder.

[ThePeach]

Also, please note that before upgrading to gulp4 I had no problems running multiple instances of gulp-responsive. Whether that's a problem specific to gulp4 or to a recent upgrade of gulp-responsive, I have no idea. I'll try to do a bit of regression testing as soon as I have some time.

In the meantime any help is greatly appreciated.

[TheDancingCode]

Did I understand correctly that you're trying to go from a folder structure like this one:

src/
  + images/
    + illustration/
    | + previews/
    |   + img1.png
    + photo/
    | + previews/
    |   + img2.png
    + pixelart/
    | + previews/
    |   + img3.png
    + vector/
      + previews/
        + img4.png

to one that looks like this?:

dist/
  + images/
    + illustration/
    | + previews/
    |   + half/
    |     + img1.png
    + photo/
    | + previews/
    |   + half/
    |     + img2.png
    + pixelart/
    | + previews/
    |   + half/
    |     + img3.png
    + vector/
      + previews/
        + half/
          + img4.png

Because that works over here. What's your paths.images.src and paths.images.dest?

[ThePeach]

currently they're set to src/images/ and dist/images/ respectively

[TheDancingCode]

I don't get it then. This works perfectly here, which is essentially the same as what I posted earlier, and generates the folder structure above:

gulp.task('buildPreviews', function() {
  return gulp.src('src/images/{photo,vector,illustration,pixelart}/preview/*.png')
    .pipe(
      responsive(
        {
          '**/*.png': [
            {
              width: 120,
              rename: {
                prefix: 'half/'
              }
            }
          ]
        },
        {
          progressive: true,
          withMetadata: false,
          errorOnEnlargement: false,
          errorOnUnusedConfig: false,
          errorOnUnusedImage: false
        }
      )
    )
    .pipe(gulp.dest('dist/images/'));
});

[ThePeach]

@TheDancingCode interesting! just tried on another machine, and it works as expected. Now I've modified another task (there's another task that does something similar but with a completely different configuration, see below) and it's not working:

function buildImages () {
  const folders = ['photo', 'vector', 'illustration'];
  const glob = `${paths.images.src}/{${folders.join(',')}}/*.{png,jpg}`;

  return gulp.src(glob)
    .pipe(responsive({
      '**/*': [ {
        width: 320,
        rename: {
          dirname: 'small',
          extname: '.jpg'
        },
        format: 'jpeg'
      }, {
        width: 768,
        rename: {
          dirname: 'medium'
        },
        withoutEnlargement: true
      }, {
        width: 1200,
        rename: {
          dirname: 'large'
        },
        withoutEnlargement: true
      } ]
    }, {
      quality: 85,
      progressive: true,
      withMetadata: false,
      errorOnEnlargement: false,
      errorOnUnusedConfig: false,
      errorOnUnusedImage: false
    }
    ))
    .pipe(gulp.dest(paths.images.dest));
}

(consider that this task has to avoid picking up anything that is not jpg or png, so I hope the combination of the glob together with the responsive config works.)

[edit] I now even started getting the segmentation fault I had at the beginning... something caching? 😞

In the previous buildPreviews task I see:

[11:47:22] gulp-responsive: illustration/preview/2012-06-27-tube-10.png -> illustration/preview/half/2012-06-27-tube-10.png

on this task (which is running separately for testing purposes) I now get

[11:56:21] gulp-responsive: illustration/2011-02-football-player.png -> small/2011-02-football-player.jpg
[11:56:21] gulp-responsive: illustration/2011-02-football-player.png -> medium/2011-02-football-player.png
[11:56:21] gulp-responsive: illustration/2011-02-football-player.png -> large/2011-02-football-player.png

(when it works)

I'm baffled.

(And clearing the node_modules directory does nothing)

[TheDancingCode]

And you would like the new task to output like this?

illustration/2011-02-football-player.png -> illustration/small/2011-02-football-player.jpg
illustration/2011-02-football-player.png -> illustration/medium/2011-02-football-player.png
illustration/2011-02-football-player.png -> illustration/large/2011-02-football-player.png

The dirname option overrides the full dirname. Use the prefix option like I did earlier. Like this:

gulp.task('buildImages', function () {
  const folders = ['photo', 'vector', 'illustration'];
  const glob = `${paths.images.src}/{${folders.join(',')}}/*.{png,jpg}`;

  return gulp.src(glob)
    .pipe(responsive({
      '**/*': [ {
        width: 320,
        rename: {
          prefix: 'small/',
          extname: '.jpg'
        },
        format: 'jpeg'
      }, {
        width: 768,
        rename: {
          prefix: 'medium/'
        },
        withoutEnlargement: true
      }, {
        width: 1200,
        rename: {
          prefix: 'large/'
        },
        withoutEnlargement: true
      } ]
    }, {
      quality: 85,
      progressive: true,
      withMetadata: false,
      errorOnEnlargement: false,
      errorOnUnusedConfig: false,
      errorOnUnusedImage: false
    }
    ))
    .pipe(gulp.dest(paths.images.dest));
});

[ThePeach]

oooooooh!!!! now I see it! I completely overlooked that! 🤦‍♂️

[ThePeach]

OK, then this is totally my fault, every time the task was moving images it was renaming the folder and killing the following copy. Dang. I'm going to close this one!

Thanks a lot for the help @TheDancingCode !!! really appreciated!