streaming stuff

[jimmywarting]

Would be cool if this included stuff that helps you stream a blob. converting the hole blob into something else is not optimal when the blob is large

this is how you would get it in the most easiest way possible

stream = new Response(blob).body

And the way you would read a stream would be to use the reader from stream.getReader()

reader = stream.getReader()
const pump = () => reader.read().then(res => {
  if (res.done) return
  console.log(res.value) // uint8array
  return pump()
})

// Start the reader
pump().then(() =>
  console.log('Done reading')
)

But there is a cooler way on the horizon. Async iterator

for async (const chunk of stream) {
  console.log(chunk) // uint8array
}
console.log('done reading')

This is thanks due to stream are getting a Symbol.asyncIterator soon

I was just wondering if you could make something like the asyncIterator possible using only the FileReader api without having to depend on Response.prototype.body that isn't available in all browser yet since it's missing ReadableStream api

so something like this

iterator = blobUtil.blobToIterator(blob, [chunksize])
for await (const chunk of iterator) {
  console.log(chunk) // uint8array
}

this could also be written as:

iterator = blobUtil.blobToIterator(blob, [chunksize])
for (const chunk of iterator) {
  console.log(await chunk) // uint8array
}

Each time you call next you would have to return a promise resolving with a uint8array

iterator = blobUtil.blobToIterator(blob, [chunksize])
chunk = iterator.next() // promise
console.log(await chunk) // uint8array (size === chunksize (if not the last))

[jimmywarting]

I believe it would work something like this:

// chunkSize is optional
blobToIterator = function* (blob, chunkSize = 524288) {
  let position = 0
  while (true) {
    const chunk = blob.slice(position, position + chunkSize)
    const promise = util.blobToArrayBuffer(blob).then(ab => new Uint8Array(ab))
    position += chunk.size
    if (position >= blob.size) return promise
    yield promise
  }
}

not tested

[nolanlawson]

My goal with blob-util is to keep things as simple as possible, but I would welcome you to write your own set of utilities to do streaming support. :)