Debouncing watcher tasks with Gulp 4

[ide]

With Chokidar there's no debounce option from what I've seen and instead it's up to the user of Chokidar to pass in a debounced function. So currently I do this:

gulp.watch('**/*', _.debounce(compile, 200));

Because lodash doesn't copy the provided function's name, my Gulp log says, "[03:06:06] Starting 'debounced'..." instead of "compile". The more serious issue is that the logs never say "Finished 'debounced'..." because the done callback that gulp.watch passed to the first invocation of the debounced function is ignored.

So what I'm looking for is an easy and reliable way to debounce gulp.watch callbacks, possibly by adding debounce support to gulp.watch itself via the options arg.

[adamreisnz]

@phated ok I'll try to do this next week

[zckrs]

@phated I just push a repository to try glob-watcher-2 branch. Repository contains a boilerplate React app with some Gulp4 Task.

https://github.com/zckrs/test-gulp-glob-watcher-2

glob-watcher seems to work properly

[phated]

@zckrs thanks! do I just switch to the 4.0 branch to see the problems outlined in this issue? I'd like to see the differences

[zckrs]

Yeah exactly like this commit https://github.com/zckrs/test-gulp-glob-watcher-2/commit/701cb190727bfd44b7db685d5477e0ac61596d9b

[phated]

@zckrs excellent, thanks again. I'll be digging into this later today!

[adamreisnz]

@phated I've implemented it in my Angular seed project as well, see https://github.com/meanie/angular-seed/tree/gulp-glob-watcher-2

You can install this and test by running npm run dev. Then, for example, change all let declarations to var within the app folder and vice versa, and see how Gulp handles the file changes.

It's much better than before, but for me, it still runs twice:

Hope it helps you with testing. Let me know if you need anything else.

[phated]

@adambuczynski the current default timeout is 100ms (looks like you were using 250) and the queue option is true, if you don't want it to queue up that second run, you would need to turn that off.

[phated]

@adambuczynski I was actually playing around with the implementation a bit before I pushed and didn't think it was working correctly, but now that I have a project to run against, it seems correct. Let me push an update and you can give it another try.

[phated]

@adambuczynski update glob-watcher with https://github.com/gulpjs/glob-watcher/commit/27f0c6b446b745c7bd64166d4c35b01ff0a0be51

[adamreisnz]

@phated seems to be working better now, it only runs once, without me needing to make any changes or pass any options:

Is it possible to mute one of the Starting 'debounced' logs? There are two of them, but only one matching "Finished" due to the debouncing.

[phated]

@adambuczynski um, you shouldn't be seeing "debounced" tasks at all, let me investigate.

[phated]

@adambuczynski I am not seeing that debounced function, are you sure you are using the branch that removed the custom debouncer?

[phated]

@adambuczynski I am seeing a queued build happening sometimes, maybe the default delay should be increased to 200ms, as I haven't seen a queued build with that adjustment.

[adamreisnz]

@phated let me double check, I updated gulp from the glob-watcher-2 branch and figured that would include the new glob watcher.

Yes, I think 200ms would be better than 100ms. I ended up with 250ms when I was playing around with it, but 200ms should be fine too as a default.

[adamreisnz]

@phated ah, was working in the master branch... sorry.

But yes, it looks good now:

🎉

[phated]

@adambuczynski excellent, and I just pushed an update for 200ms default delay. If things are looking good on your end, I should get started on writing tests soon.

[adamreisnz]

@phated looks good with the seed project. If you give me a couple of days, I will also try to test it in a larger project, with more files. See how it performs there. I can give you an update on that next Thursday.

[phated]

@adambuczynski perfect, I should have time before then to write tests and with the culmination of everything, we should be able to get it into the gulp 4 branch.

[phated]

@adambuczynski friendly reminder to follow up when you have a chance. I've been working on tests locally and should have them done this week.

[adamreisnz]

@phated thanks, I also put a reminder in place for myself for this week :)

[adamreisnz]

@phated seems to work well. I did a replacement across 75 files, and Gulp waited patiently until all replacements were done before executing the tasks. The delay time seems to be spot on.

[phated]

This is complete as of https://github.com/gulpjs/gulp/commit/6c03475e1a39d18c139b4d56baa1c14a587f9f4a

Thanks to everyone that helped, tested or gave other insights.

[adamreisnz]

Great work, thank you. Any idea when gulp 4 will be coming out of alpha?

[phated]

@adambuczynski very soon. Need to firm up vinyl-fs first

[tkodev]

I understand this is an old thread but anytime I google "debounce gulp watch" or "gulp watch runs multiple times", this thread comes up. So here is what I did for anyone reading. What I did to avoid bouncing in my situation was use change instead of all in:

gulp.task("watch", function(done) {
  gulp.watch( ["sass/*.sass", "build/*"] )
    .on( "change", gulp.series("sass", "build", "lint", "reload") );
});

• The application only has one asset folder. sass deletes css files in build/ and copies files from sass/*.sass + creates a sourcemap per file (total of 2 deletions + 2 new files = 4 file operations).
• build then uses build/* to do its work. There are other assets in build/ as well hence why I must watch both.
• all will trigger anytime files are deleted then added in the build/ folder which means it triggers 4 times (see above)
• change works around by triggering only when files are modified (no trigger on file creation / deletion).

That being said, now I can't watch for new files, but its a compromise I can deal with...

Its probably best practice to separate your watched folders from output folders when possible to avoid feedback loops. Unfortunately, while it might prevent looping, using all will always bounce in my situation because task sass creates so many file operations. Why gulp.watch picks up these operations during the same task execution is confusing. Any suggestions?