akoenig / gulp-svg2png

A gulp plugin for converting SVGs to PNGs.
MIT License
54 stars 34 forks source link

Max user processes #7

Closed felixakiragreen closed 8 years ago

felixakiragreen commented 10 years ago

Has anyone else hit the max process limit? I have 500 svgs that I export twice (@2x) and it bring my Mac to a grinding halt and after a few minutes I receive a Error: spawn EAGAIN.

screen shot 2014-11-07 at 14 03 47

Please note, this is not the max file limit. I have found many questions and answers about increasing the files open limit. This is related to have too many phantomjs processes running. In terminal ulimit -a tells me that my max user processes is 709. I don't want to change this. I just want it to not try and be exceeded.

mediafreakch commented 9 years ago

I have a similar problem, when processing a folder with ~100 svgs. The error message in my case is:

stream.js:94
      throw er; // Unhandled stream error in pipe.
            ^
Error: spawn EMFILE

I'm working on a Mac too. Is it something the plugin maintainer can fix or is it related to another plugin?

nihaopaul commented 9 years ago

same issue, around 160 SVGs, basically need a way to limit the amount of concurrent processes, personally i'd love to run a limited amount of concurrent processes

felixakiragreen commented 9 years ago

I have found a temporary solution is to manually limit them with a timeout and loop. But this is tedious and NOT the right way to go about it.

nihaopaul commented 9 years ago

i have a solution but its built against 0.3.0, not sure how to share, i'm trying to submit against Master branch but suffering with typescript :'(

nihaopaul commented 9 years ago

the meat of it is as follows: replace: return map(..)

with

  var files = [];

  return es.through(function write (data) {
    files.push(data);
  }, function end () {

    async.eachLimit(files, concurrency || 4, function (file, cb) {
      convert(file, cb);
    }, function (err) {
      this.emit('end', err);
    }.bind(this));
  });

replace map-stream with es = event-stream and install async

once i figure out my failure with typescript i will submit a PR

oh and the module.exports line should be:

module.exports = function (scale, verbose, concurrency) ...

yes, concurrency as a variable, maybe time to move to {options}

WIP so far: https://github.com/nihaopaul/gulp-svg2png/tree/feature/parallel-tasks

sillero commented 9 years ago

This PR worked good for me. Would be nice to have it merged. Currently using gulp-iconify which has gulp-svg2png as dependency.

akoenig commented 8 years ago

Fixed in 1.0.0 :)