Closed MarcoPolo closed 4 months ago
Hi! Having Milo running on browsers it was already on my roadmap, but I was currently focusing on bringing Milo into Node internals without sacrificing perfomance. Anyway I think your PR looked fine, thanks a lot for this!
@MarcoPolo @ShogunPanda Can you provide some instructions for using this, say when the incoming stream is Uint8Array
. Thanks in advance!
Here's how I use it to read an HTTP message from a Duplex<Uint8Array>
which, for reading, is basically a generator that returns Uint8Arrays. https://github.com/libp2p/js-libp2p-http-fetch/blob/main/src/fetch/index.ts#L97
@CxRes The example @MarcoPolo already contains that.
If you want a even more isolated example, you think about something this for Node streams:
const milo = require('@perseveranza-pets/milo')
// Get the stream somehow
const stream = getStream()
// Declare a chunk pointer. We need in callbacks
let lastChunk
// Create a parser
const parser = milo.create()
// Declare our callbacks
milo.setOnData(parser, (p, from, size) => {
console.log(`Pos=${milo.getPosition(p)} Body: ${lastChunk.slice(from, from + size).toString()}`)
})
// Here we're passing the chunk directly, which will copy the data.
// In the real world it's better to use WASM backed buffers allocated with milo.alloc if possible.
// Unless specify differently, a stream is always a sequence of Uint8.
stream.on('data', chunk => {
lastChunk = chunk
const consumed = milo.parse(parser, chunk, chunk.length)
// You should check here if all bytes were consumed.
})
I think you can easily adapt this example to WebStream. Is that enough?
@MarcoPolo @ShogunPanda Thank a bunch for your responses!
I think you can easily adapt this example to WebStream. Is that enough?
Not sure, though adapting it to a webstream is not a problem, I will have to give it a try when I get a chance. What I am trying to do is read a multipart (or possibly application/http in a later iteration) stream and would want to it break up in sequence of Fetch Response objects for each part body (which are little http like messages). So I want to be able to parse headers (don't need trailers) and pass on the little bodies as streams to the consumer. The important thing is I do not want chunk bodies forcing the consumer to wait for an entire part.
I am doing this for a new protocol I am developing on HTTP for notifications called Per Resource Events. I am coming out with libraries for it and would be better able to show you what I mean once they online.
Hey Paolo, thanks for making this. It's just what I needed for the js-libp2p implementation of libp2p+HTTP (tl;dr - partial summary - run HTTP request/response on top of libp2p streams).
I got milo working as the HTTP parser in the browser, with a couple of changes to use TypedArrays instead of the Node-only
Buffer
. It's not too much work.Would you be open to merging a change to support browser (and non-node) environments?
I see a couple of options:
Buffer.from
and check fortypeof Buffer !== 'undefined'
.