Closed domenic closed 8 years ago
Updated domenic's first comment to use as a task list.
I started on this a bit by documenting some recently-discovered constraints in Requirements.md, although it was pretty quick and not too comprehensive.
I'd like to prototype the unified byte stream underlying source idea we discussed today.
ReadableByteStream
has a queue [[queue]]
[[queue]]
is not empty, autoReader.read()
gets fulfilled with the ArrayBufferView
at the top of the [[queue]]
[[queue]]
is empty, autoReader.read()
gets queued in the [[pendingReads]]
queuesource.pull()
is called to notify that a new read()
is made on the autoReader
source
calls delegate.enqueue()
to enqueue an ArrayBufferView
to the [[queue]]
[[queue]]
is not empty, when byobReader.read(view)
is called, view
gets filled with bytes on the ArrayBufferView
s in the [[queue]]
. If the bytes are not sufficient to fill up view
fully, source.read(newView)
is called where newView
specifies the not yet filled region[[queue]]
is empty, byobReader.read(view)
calls source.read(view)
source.read(view)
may be called in a row without waiting for the previous source.read(view)
getting fulfilledpull()
is not called after source.read(view)
until source
fulfills the source.read(view)
. This is guaranteed by the requirement that a byob reader cannot be released until the pending reads are all fulfilledsource.read(view)
is fulfilled by calling delegate.done()
. Not delegate.enqueue()
.source calls delegate.enqueue() to enqueue an ArrayBufferView to the [[queue]]
I wonder if we should have the stream implementation enforce the type as Uint8Array.
If the bytes are not sufficient to fill up view fully, source.read(newView) is called where newView specifies the not yet filled region
Alternately, we could just fill up view as much as possible, and have the returned ArrayBufferView be of a shorter byteLength. Just like if read(2) returns a smaller number of bytes than were passed in.
The more interesting case is: if [[queue]] is not empty, but its first chunk is of a larger size than view
. Then we need to copy part of that chunk into view
and leave the rest in the queue---at least, if implemented in JS. (C++ implementers might have more tools available, to avoid copies, and instead just make the returned Uint8Arrays map to the right memory regions.)
Agreed with everything else. Good detailed analysis.
It's interesting how much shared infrastructure there will be between ReadableByteStream and ReadableStream. I wonder if ReadableByteStream does end up just being a subclass after all. Hmm.
I wonder if we should have the stream implementation enforce the type as Uint8Array.
Added to #300.
Alternately, we could just fill up view as much as possible, ...
Sounds good. Added to #300.
The more interesting case is: if [[queue]] is not empty, but its first chunk is of a larger size than view. ...
This makes sense for the non-BYOB reader if we add a size-limiting parameter to its read()
method as you discussed last week.
For BYOB, anyway we need to append the data to the given buffer as the consumer may already have written some data in the encapsulated ArrayBuffer and expect the new data to be added right next to the data the region of which is specified by the given view.
Another point I wonder how we should deal with is, whether the partially consumed bytes in the large buffer should be considered as freed or not from backpressure point of view. If we cannot free the partially consumed region, it continues to have memory pressure. So, should we refrain from pulling?
Sorry for the delay on replying to this. It slipped off my to-do list somehow.
Another point I wonder how we should deal with is, whether the partially consumed bytes in the large buffer should be considered as freed or not from backpressure point of view. If we cannot free the partially consumed region, it continues to have memory pressure. So, should we refrain from pulling?
I think we should be able to free the consumed region. Even in JavaScript you could ArrayBuffer.transfer to a smaller buffer which would let the GC collect the consumed part.
This seems to be pretty well taken care of in the latest spec, with good examples and interop.
A few things I can think of worth including: