holepunchto / hypercore

Hypercore is a secure, distributed append-only log.
https://docs.holepunch.to
MIT License
2.58k stars 182 forks source link

feed.createReadStream is not async iterable? #272

Closed timcash closed 4 years ago

timcash commented 4 years ago

Should feed.createReadStream be async iterable? This would allow code like

async function logChunks(stream) {
    for await (const chunk of stream) {
        console.log(chunk);
    }
}
tinchoz49 commented 4 years ago

Could be a nice thing to have. Right now createReadStream use from2 to build the readable stream but that module uses an old version of readable stream without the support for iterators.

We could update the dependencies and/or also we can add a new method to provide an iterator feed.iterator() directly without having to use streams.

RangerMauve commented 4 years ago

I usually pipe streams from libraries like hypercore through a passthrough stream since they mostly depend on readable-stream where this async stuff is still a WIP. 😅

tinchoz49 commented 4 years ago

Me too (I am used to it) but async iterators streams is already stable in node v12, you can do:

const { Readable } = require('stream')

async function * generate () {
  yield 'a'
  yield 'b'
  yield 'c'
}

const readable = Readable.from(generate())

;(async () => {
  for await (const chunk of readable) {
    console.log(chunk)
  }
})()
RangerMauve commented 4 years ago

Yeah, honestly working with async iterators has been a lot nicer than regular streams in my experience. Especially for object streams. Been using it with hyperswarm in p2plex and would want to do so here, too.

RangerMauve commented 4 years ago

BTW, this is how you can get it to work with your example:

const { PassThrough } = require('stream')

async function logChunks(stream) {
    for await (const chunk of stream.pipe(new PassThrough())) {
        console.log(chunk);
    }
}
tinchoz49 commented 4 years ago

oh that's a really cool solution, I didn't understand you before.

mafintosh commented 4 years ago

You’d wanna pump for error handling there. We are gonna switch all the streams over to streamx

mafintosh commented 4 years ago

now async iterable thanks to @martinheidegger

p3nGu1nZz commented 3 years ago

very nice design pattern. thank you for contributing. have a cookie🍪 or 🍪 🍪