yuanchuan / node-watch

A wrapper and enhancements for fs.watch
https://npm.im/node-watch
MIT License
339 stars 44 forks source link

Some events are lost when too many are triggered #91

Closed blarfoon closed 4 years ago

blarfoon commented 4 years ago

This works perfectly with a low amount of events, but if I try to copy 200+ files into the watched directory, some of the files will not trigger the callback. I can consistently reproduce this issue, but every time the number of files that trigger the callback is different. Usually, it's around 150 out of 200. UPDATE: It looks like copying files work. Just when I move them it skips some of them. Maybe they're too fast?

yuanchuan commented 4 years ago

Is there a code sample and a sequence of commands which caused the problem?

blarfoon commented 4 years ago
var watch = require('node-watch');

watch('./dir', { recursive: true }, function(evt, name) {
  console.log('%s changed.', name);
});

Something like this. Then just start cutting and pasting like 300 files and you should be able to reproduce it.

yuanchuan commented 4 years ago

OK. I need more information. I can't reproduce it on my computer (Mac, Node v11.7.0, node-watch 0.6.3)

  1. How do you test if some of the files do not trigger the callback?
  2. Does it happen on cutting or pasting or both?
  3. Are there nested directories among the cutting/pasting files?
crh3675 commented 4 years ago

Sounds like a system limit like ulimit or fs.max-open-files issue

blarfoon commented 4 years ago

I test it counting the files that I'm copying, and then implementing a counter in the listener. This just happens on cutting as far as I know. No nested directories. Chokidar and NSFW works perfectly fine

yuanchuan commented 4 years ago

I wrote a test and nothing went wrong on Travis. Also, I managed to find a Desktop Windows to do 'cutting and pasting' of 300 files but I couldn't reproduce it either.

There's a delay (200ms) to respond to an event. Maybe there's something wrong about the counting method.

May I know how did you implement the counter?

yuanchuan commented 4 years ago

Or try both node-watch with option{ delay: 0 } and native fs.watch() to see if there's any difference

yuanchuan commented 4 years ago

Thank you @killpowa for reporting. I'll close it by now. Also thanks to @crh3675 for the suggestion :)