tc39 / proposal-async-iterator-helpers

Methods for working with async iterators in ECMAScript
https://tc39.es/proposal-async-iterator-helpers/
95 stars 3 forks source link

concurrent versions of `forEach`/`some`/`every`/`find`? #6

Open bakkot opened 1 year ago

bakkot commented 1 year ago

These aren't strictly necessary given a bufferAhead helper, because you can always replace .some(pred) with .map(pred).bufferAhead(3).some(x => x) (or similar). But that's kind of gross.

I definitely don't want to add a "concurrency" parameter to the iterator-producing helpers, since it's much easier to use when concurrency of an iterator chain is determined by how many times the consumer pulls from it. (Some more discussion of that in the second half of this comment.)

But these methods are the consumer, and might themselves be async and able to run their functions concurrently. Should we have concurrent versions of the consumers?

benlesh commented 1 year ago

RxJS (and other observable implementations) implement forEach to take a callback and return a Promise<void> when it's done.

I definitely don't want to add a "concurrency" parameter

I would avoid concurrency parameters in EVERYTHING if possible. RxJS exposes a "concurrency" parameter in our mergeMap operator and it's very rarely used or useful.

laverdet commented 4 months ago

Could bufferAhead pass along information in the derived iterable which is used to signal ahead that the iterable is concurrent? Something like { [Symbol.asyncConcurrency]: 3, next() { ... } }. Folding operations could use that signal to evaluate concurrently.

It definitely gives me the icks but given the strong hesitation against adding concurrency parameters it seems like one of few choices.

bakkot commented 4 months ago

bufferAhead already passes concurrency information to the iterator it's consuming simply by calling next multiple times concurrently. And passing the information to the iterator that's consuming bufferAhead isn't so useful: you wouldn't want iter.bufferAhead(N).forEach(foo) to call foo concurrently; that's something that the person calling forEach would need to opt in to.