Does cache clear upon termination of gulp task?

[cresharper]

I have an images task that pipes/minifies etc to an images folder along with automated image optimization. I was experiencing a performance problem where if I only changed/deleted/added 1 image, the entire library would upload. I switched from gulp-changed to gulp-cached which is looking promising, but there is one catch...it seems my cache clears every time I terminate and restart my gulp watch task.

FWIW my code/task looks like this:

var imagesTask = function() {
  var flyoutResizeFilter = gulpFilter('**/*/accessories-*png', {restore: true});
  return gulp.src(paths.src)
     // below used to be .pipe(changed(paths.dest))
     .pipe(cached(paths.dest)) // Ignore unchanged images
     .pipe(imagemin({
       plugins: [
         imagemin.jpegtran(), imagemin.gifsicle(), imagemin.optipng({optimizationLevel:5}), imagemin.svgo()
       ],
       verbose: true,
     }))
    .pipe(flyoutResizeFilter)
    .pipe(imageResize({
      height: 135
    }))
    .pipe(flyoutResizeFilter.restore)
    .pipe(rename({
      prefix: 'images-'
    }))
    .pipe(gulp.dest(paths.dest))
}

If I add an image, the entire library uploads - once that is done if another image is changed or added it behaves as expected. So this is more of a 2 part question:

1. Does the cache clear every time a gulp process is terminated?
2. If so is there a way to preserve it after termination?

[yocontra]

No, cache does not terminate unless you explicitly clear it. It last forever as long as the process stays alive. If you need to persist it between processes you can write it to a json file and load it again on start.

[cresharper]

@contra thanks for getting back to me - I would like to have it persist between processes and even after the process gets terminated. What would be the best way to structure my json file to do this?

[andrewchilds]

Here's a reference implementation (using gulp-eslint), in case it helps anybody else:

const fs = require('fs');
const gulp = require('gulp');
const eslint = require('gulp-eslint');
const gulpIf = require('gulp-if');
const gulpCached = require('gulp-cached');

// fs functions are relative to the cwd.
const CACHE_FILE = './.cache.eslint.json';

// require() is relative to the current file.
// This assumes you're running `gulp watch` from the root directory,
// and this file is located in build/tasks/linter.js.
const REQUIRED_CACHE_FILE = '../../' + CACHE_FILE;

// Replace with however you determine dev environment:
function isDevelopment() {
  return true;
}

if (fs.existsSync(CACHE_FILE)) {
  console.log('Using eslint cache file at ' + CACHE_FILE);
  gulpCached.caches = require(REQUIRED_CACHE_FILE) || {};
} else {
  console.log('No eslint cache file found.');
}

function linter() {
  var stream = gulp.src('app/**/*.js')
    .pipe(gulpIf(isDevelopment, gulpCached('eslint')))
    .pipe(eslint({ useEslintrc: true }))
    .pipe(eslint.formatEach())
    .pipe(eslint.failAfterError());

  stream.on('end', function () {
    _saveCache();
  });

  // Not sure if we need to do this, but just in case:
  stream.on('error', function () {
    _clearCache();
  });

  return stream;
}

function _saveCache() {
  var json = JSON.stringify(gulpCached.caches, null, '  ');
  fs.writeFileSync(CACHE_FILE, json);
}

function _clearCache() {
  fs.writeFileSync(CACHE_FILE, '{}');
}

module.exports = linter;

[yocontra]

@andrewchilds Thanks for posting the reference, I'm sure that will help people a lot. Might make a nice little module, I can add it to the README if you end up making it.

[svivian]

Would be nice if there was an option for saving the cache file to disk in gulp-cached. Surely it must be a common occurrence that you modify files when not running gulp watch?

Also as a heads up to anyone else using @andrewchilds solution above, I'd strongly recommend using {optimizeMemory: true} as otherwise it saves the entire contents of each file in your cache. On a folder with 8.7 MB of images my cache file was over 18 MB. Using optimizeMemory it's 41 KB. And that's only one folder, I have over 500 MB in other folders.

Here's my entire task, it's a bit simpler than above so maybe it will help others:

gulp.task('images', function() {
    const imagemin = require('gulp-imagemin');
    const fs = require('fs');
    const gulpCached = require('gulp-cached');

    const imageDir = '/path/to/images';
    const cacheFile = './.gulp-cached.json';

    gulpCached.caches = require(cacheFile) || {};

    var stream = gulp.src(imageDir + '/**')
        .pipe(gulpCached('images', {optimizeMemory: true}))
        .pipe(imagemin(/* options */))
        .pipe(gulp.dest(imageDir));

    stream.on('end', function () {
        var json = JSON.stringify(gulpCached.caches);
        fs.writeFileSync(cacheFile, json);
    });

    return stream;
});

[yocontra]

@svivian I'd be open to it if you sent a PR - my only hesitation is sync writing blocking the loop. I'm also thinking I might make optimizeMemory the default, it incurs almost no overhead to do the hashing. I think the hashing in node used to be slower when I made this module so it was optional.