Open johnv-git opened 8 years ago
Can you try with the latest version (0.1.7)? It includes some performance updates.
Much improved in 0.1.7. Looks to be 4x faster than 0.1.6. I still have idle cpu, so there is room for even more improvement. Perhaps an option to select the number of concurrent threads? (Assuming gm allows multiple concurrent invocations.)
I took a look at the async routines and tuned the cpu usage by using a combination of series() and parallelLimit(). So where you accumulate the resize tasks into series
, I'm accumulating them into resizeseries
and then parallelize that at some concurrency level. 3 runs nicely on my box without pegging the cpu.
In practice its a one-time hit as long as you write to a tmp dir that isn't cleaned between rebuilds and use options.newFilesOnly = true. But maybe this helps someone?
// Run resize operations in parallel at a specified concurrency.
options.concurrency = 3;
series.push(function(callback) {
async.parallelLimit(resizeseries, options.concurrency, callback);
});
series.push(function(callback) {
outputResult(tally[sizeOptions.id], sizeOptions.name);
return callback();
});
Here's a patch that I'm running locally. Works nicely with concurrency set to 3.
--- node_modules/grunt-responsive-images/tasks/responsive_images.bak 2015-10-05 18:03:11.564669200 -0700
+++ node_modules/grunt-responsive-images/tasks/responsive_images.js 2015-10-10 01:44:57.365276300 -0700
@@ -35,6 +35,7 @@
customIn: null,
customOut: null,
sharpen: null,
+ concurrency: 1, // Simultaneous gm processes to invoke. CPU Cores - 1 is a reasonable choice.
sizes: [{
name: 'small',
width: 320
@@ -470,6 +472,7 @@
options.units = _.extend(_.clone(DEFAULT_UNIT_OPTIONS), options.units);
options.sizes.forEach(function(s) {
+ var resizeseries = [];
var sizeOptions = _.extend({}, options, s);
@@ -512,7 +515,7 @@
// remove pixels from the value so the gfx process doesn't complain
sizeOptions = removeCharsFromObjectValue(sizeOptions, ['width', 'height'], 'px');
- series.push(function(callback) {
+ resizeseries.push(function(callback) {
if (sizeOptions.newFilesOnly) {
if (isFileNewVersion(srcPath, dstPath)) {
@@ -528,6 +531,10 @@
});
});
+ // Run resize operations in parallel at a specified concurrency.
+ series.push(function(callback) {
+ async.parallelLimit(resizeseries, options.concurrency, callback);
+ });
series.push(function(callback) {
outputResult(tally[sizeOptions.id], sizeOptions.name);
return callback();
Added as pull#99
Running on an i7 Windows 7 box, this runs cpu at 20%. It'd be nice to use all available cpu and finish 5x faster. But perhaps a gm limitation?
It looks like grunt-imagemin pegs the cpu, so whatever they're doing in terms of pipeline seems like the right thing.