Example of deferred.gate with file recursion

[madarche]

With this example I'm interested in both contributing and having a check on the way it is done.

It's very important for me that the indexDocs method returns a promise so that it can be chained later on.

Cheers!

[medikoo]

@madarche I decided not take, as it's actually just enriched example that we already have.

However I took your points, and improved existing example so it covers all you wanted to add here:

1. I turned an example into a reusable module (which you can use as indexDocs function)
2. I added descriptive note about readdir from fs2 package, as indeed it's a savior in many cases.
3. I added note about workarounds for EMFILE problem (deferred.gate is not the only solution)
4. I've also made sure we read stats file data into our results.

See: https://github.com/medikoo/deferred/blob/master/examples/process-directory-of-html-files.js I think it's way better that way.

Thank you!

[madarche]

Thanks @medikoo ! Better file name, helpful comments and much easier to reuse.

Could I request your help a little bit more to have the proposed readFirstBytes method reviewed and hopefully integrated?

Cheers

[medikoo]

@madarche Sure, however I think such function should internally be stream based (so we don't read full file content just to get first few bytes), and promise usage may be very basic in such case (create, return promise and resolve).

[madarche]

@medikoo I haven't used stream so far, because this Node API is marked unstable at the moment. But if you think it's a good idea, I would be delighted to learn from your code!

[medikoo]

e.g. this is where I try to read shebang from file -> https://github.com/medikoo/xlint/blob/master/lib/lint-files.js#L69-L75

I'm actually using fs.read just to get first 100 characters, but probably it can also be done via fs.createReadStream. Latter might be better if we don't know number of characters upfront but want to decide whether to continue after examining already retrieved content.