Open bakkot opened 1 year ago
RxJS (and other observable implementations) implement forEach
to take a callback and return a Promise<void>
when it's done.
I definitely don't want to add a "concurrency" parameter
I would avoid concurrency parameters in EVERYTHING if possible. RxJS exposes a "concurrency" parameter in our mergeMap
operator and it's very rarely used or useful.
Could bufferAhead
pass along information in the derived iterable which is used to signal ahead that the iterable is concurrent? Something like { [Symbol.asyncConcurrency]: 3, next() { ... } }
. Folding operations could use that signal to evaluate concurrently.
It definitely gives me the icks but given the strong hesitation against adding concurrency parameters it seems like one of few choices.
bufferAhead
already passes concurrency information to the iterator it's consuming simply by calling next
multiple times concurrently. And passing the information to the iterator that's consuming bufferAhead
isn't so useful: you wouldn't want iter.bufferAhead(N).forEach(foo)
to call foo
concurrently; that's something that the person calling forEach
would need to opt in to.
These aren't strictly necessary given a
bufferAhead
helper, because you can always replace.some(pred)
with.map(pred).bufferAhead(3).some(x => x)
(or similar). But that's kind of gross.I definitely don't want to add a "concurrency" parameter to the iterator-producing helpers, since it's much easier to use when concurrency of an iterator chain is determined by how many times the consumer pulls from it. (Some more discussion of that in the second half of this comment.)
But these methods are the consumer, and might themselves be async and able to run their functions concurrently. Should we have concurrent versions of the consumers?