In some cases, doesn't fire when more than 16 files get processed

[JacobDB]

This module works great for most of my tasks, but for some reason it doesn't appear to be working correctly with my upload and rsync tasks.

My default gulp task runs through my html, styles, scripts, and media tasks, and accepts a flag --upload or -u to optionally upload files via FTP (sftp, ftp, or ftps). I use gulp-notify to mark when each task is complete, and it works great for my four main tasks. When it gets to the upload task, however, the notification never fires. I've tried removing onLast, and it does then trigger, but only 16 times, when there are 50+ files that are being uploaded.

I'm not sure what's happening (unfortunately I'm not fantastic with Node JS), but it seems to me like it's for some reason breaking after a certain point in the stream. I'm probably doing something wrong, but I'd greatly appreciate some help.

• gulpfile.js
• upload.js
• rsync.js

I'm having similar trouble with my rsync task, but that one's harder for me to test for various reasons, so I'm primarily focusing on the upload task for this issue, as I suspect they're related.

[JacobDB]

I've verified that this appears to occur when there are more than 16 files being uploaded. I can reliably get the notification to trigger with 16 files, but as soon as a 17th file is added, it doesn't show the notification.

I'll try and replicate with my rsync task tonight.

[JacobDB]

I was able to replicate this with my rsync task, it's for some reason quitting after exactly 16 files. I don't see any reason why this would be.

[JacobDB]

Possibly related:

• https://github.com/adametry/gulp-eslint/issues/36
• https://github.com/rvagg/through2/issues/82

[JacobDB]

Figured it out! I'm not great at Gulp/Node JS, but I guess I needed to "consume the stream." This normally happens when using gulp.dest(), but because these two tasks didn't have a dest(), the stream was never getting consumed. I found a module stream-consume, and used that in conjunction with through2 to consume the stream after files have been uploaded/synced.

• https://github.com/gulpjs/gulp/tree/master/docs/writing-a-plugin#streaming-file-objects
• https://github.com/aroneous/stream-consume
• https://github.com/rvagg/through2

[mikaelbr]

Hi! Good investigation on this. FYI, I also think you can just add .on('data', noop) as a sink at the end. Where noop is a empty function. As I remember it, this will cause the stream to fallback on a push stream and start consuming data as you go.

[JacobDB]

Ah, thanks, I knew there must've been a way to do it without adding another module!