search

medikoo / deferred

Modular and fast Promises implementation for JavaScript
ISC License
364 stars 20 forks source link

Max amount of async calls #7

Closed jansmolders86 closed 11 years ago

jansmolders86 commented 11 years ago

Hi there,

You helped on stackoverflow with this issue: http://stackoverflow.com/questions/14883472/javascript-nodejs-different-async-functions-resulting-in-one-big-json-file/14890620#comment20905823_14890620

I hope you don't mind me asking a bit more about this but even though the script worked perfectly with a couple of files, I now hooked it up with several hundreds of files. But strangely the script simply stops between 50 and 60 calls without giving an error or continuing. Could you shed some light why this occurs?

snrmwg commented 11 years ago

Are you running under linux? Maybe you hit a ulimit of open files? Try a google search for "ulimit open files" to get more infos. Async library has an option to limit maximum parallel tasks to prevent hitting such limits. Please ask if you need more details. Greetings, Marc (snrmwg@gmail.com)

Am 18.02.2013 um 14:49 schrieb jansmolders86 notifications@github.com:

Hi there,

You helped on stackoverflow with this issue: http://stackoverflow.com/questions/14883472/javascript-nodejs-different-async-functions-resulting-in-one-big-json-file/14890620#comment20905823_14890620

I hope you don't mind me asking a bit more about this but even though the script worked perfectly with a couple of files, I now hooked it up with several hundreds of files. But strangely the script simply stops between 50 and 60 calls without giving an error or continuing. Could you shed some light why this occurs?

— Reply to this email directly or view it on GitHub.

medikoo commented 11 years ago

@jansmolders86 Firstly, if you don't see any error but error occurs, it means there's a problem with your flow, are you sure all promise chains are ended with end?

Secondly descriptors limit problem (mentioned by @snrmwg) can be easily solved with e.g: https://github.com/medikoo/fs2#descriptorshandler

Still, in first place I would definitely focus on fixing the flow and exposing an error (if indeed one occurs), this will confirm what's the real problem, otherwise we're just guessing.

medikoo commented 11 years ago

@jansmolders86 I'm closing this, as it's not deferred issue. If you still have problem with this, just please provide exact code example on deferred's mailing list