In #4 has the idea of a bufferAhead helper that can create concurrency. It has some discussion about ways it might want to limit concurrency, & in that PR I've advocated for keeping bufferAhead simpler / complecting less concerns into it. But the need seems real to create a way to limit concurrency (the number of unresolved promises at any given time).
let pages = asyncIteratorOfUrls
.map(u => fetch(u))
.limitConcurrency(2);
// bad consumer we want to guard against.
// more likely examples might be some kind of a thread pool.
for (let i = 0; i < 10; ++i) {
pages.next().then(() => console.log("got page"));
}
There's a number of jobs where I've created or been around folks making Staged Event Driven Architecture (SEDA) like structures, where there's a bunch of processors & queues in between them. Along with bufferAhead, the limitConcurrency(n) helper proposed here should allow fulfillment of those architectures quite quickly:
It's possible we could try to combine all these behaviors into one big helper, and the code above definitely looks a bit simple. But de-complecting the concerns feels like it creates more possibilities than trying to tackle everything together.
In #4 has the idea of a
bufferAhead
helper that can create concurrency. It has some discussion about ways it might want to limit concurrency, & in that PR I've advocated for keepingbufferAhead
simpler / complecting less concerns into it. But the need seems real to create a way to limit concurrency (the number of unresolved promises at any given time).There's a number of jobs where I've created or been around folks making Staged Event Driven Architecture (SEDA) like structures, where there's a bunch of processors & queues in between them. Along with
bufferAhead
, thelimitConcurrency(n)
helper proposed here should allow fulfillment of those architectures quite quickly:It's possible we could try to combine all these behaviors into one big helper, and the code above definitely looks a bit simple. But de-complecting the concerns feels like it creates more possibilities than trying to tackle everything together.