Closed boneskull closed 4 years ago
cc @mcollina
So … basically, the desired feature here is that stream.pipeline()
should transparently convert async generators to object-mode Transform
streams?
@addaleax I guess that'd be pretty great. Readable#pipe
too, maybe?
something something migration path from streams to async iterators and generators
if we can coerce async iterables to streams, that would enable some nice compat with whatwg streams. +1 from me
How can we coerce to a transform? What signature would you expect from a transform async iterator?
@mcollina I think any function that takes one argument, which is an async-iterable for Buffers or strings, and returns an async-iterable of that kind should work?
More generally, I think we could create an utility class that turns async-iterables of Buffers/strings into a Readable
stream, and use that as a building block for the .pipe()
utility.
https://github.com/mcollina/stream-iterators-utils/blob/master/README.md#utilstoreadablegenerator-opts this is a prototype implementation of that Readable pattern.
It's awesome that we now have streams.Readable.from()
and readable[Symbol.asyncIterator]
, the only piece missing in the puzzle now is streams.Transform.from()
with a signature like
class Transform<I, O> {
static from<I, O>(generatorFn: (input: AsyncIterable<I>) => AsyncIterable<O>): Transform<I, O>
}
We added support for async generators in pipeline in v13.x. I believe that full-fills this topic. Please re-open of there is anything further to address.
Apologies for all the code here.
We can consume a
Readable
stream using an async iterator:And we can use async generators similarly to how one would use a
Transform
stream:Then, we can "pipe" these together like so:
That's neat, but also kind of hideous. What if we could leverage
stream.pipeline()
to do something like this?I'm unfamiliar with the guts of
stream.pipeline()
--and completely new to async iterators and generators--so don't know how feasible something like this is.FWIW, the "hideous nested function calls" can be naively replaced by use of the godlike
Array.prototype.reduce()
:Reference: https://twitter.com/b0neskull/status/1115325542566227968