appnexus / sicksync

Don’t accept the available as the preferable. Go the extra mile with extra speed.
Apache License 2.0
68 stars 11 forks source link

Excludes aren't 100% excluded #47

Closed laggingreflex closed 7 years ago

laggingreflex commented 7 years ago

It seems ignored paths aren't 100% ignored. Like if I change something in the .git directory, surely it doesn't actually get uploaded but it still triggers a file change event, meaning it's still being monitored which is unnecessary. Having a huge node_modules or otherwise ignored folders cause it to take up considerable time to start up.

This can be tested by putting a console log here: src\local\fs-helper.js and changing any file in one of the ignored folders, .git/info/excludes

laggingreflex commented 7 years ago

It seems the cause of this is the format of exclude patterns. .git/**, and .git/**/* don't work, but what worked for me was **/.git. It has something to do with how chokidar/anymatch parses and matches these ignore patterns...

I should note that it only adversely affects the startup performance, not the general/core functioning. Since I see that internally this is not just relying on above modules and actually reevaluating ignored paths (like in onFileChange above). Rsync too works great with these patterns. It's only the filewatcher that's causing performance issues.

laggingreflex commented 7 years ago

I really would've liked if simply .git, or node_modules had worked as they do with git in .gitignore

Would it be desirable to augment this kind of ignore pattern parsing further? I know it already works as far as the core functionality of the project is concerned but it really shoots up the startup time for large ignored folders...

laggingreflex commented 7 years ago

I've made a PR to chokidar that basically prefixes **/.git to paths which can solve this. Wrong approach. Better fix was to just add cwd option

joelgriffith commented 7 years ago

@laggingreflex I think your PR satisfies this issue...

joelgriffith commented 7 years ago

Released in 3.0.0, thanks!