Unfortunately, the built-in tee algorithm backpressures to the faster consumer and enqueues chunks without limit to the slower consumer. Therefore, it is generally not safe to use when the two branches are read by independent consumers at different speed, since the consumer will eventually run out of memory.
I believe that this was an unfortunate design mistake because it conflicts with the suggestion in Stream requirements:. You must be able to pipe a stream to more than one writable stream. which said, “The tee stream can use a number of strategies to govern how the speed of its outputs affect the backpressure signals it gives off, but the simplest strategy is to pass aggregate backpressure signals directly up the chain, thus letting the speed of the slowest output determine the speed of the tee.” This also differs from analogous operations in other libraries such as repeated nodejs Readable.pipe or akka-streams Source.alsoTo which backpressure to the slower consumer and only require a bounded queue.
But at least we can clarify the informational text, which implied that it is safe to tee() a stream and consume it independently. It is generally not safe to consume a tee()’d stream independently unless the difference in consumed data between the faster consumer and the slower consumer fits in memory.
[ ] At least two implementers are interested (and none opposed):
[ ] Tests are written and can be reviewed and commented upon at:
Unfortunately, the built-in tee algorithm backpressures to the faster consumer and enqueues chunks without limit to the slower consumer. Therefore, it is generally not safe to use when the two branches are read by independent consumers at different speed, since the consumer will eventually run out of memory.
I believe that this was an unfortunate design mistake because it conflicts with the suggestion in Stream requirements:. You must be able to pipe a stream to more than one writable stream. which said, “The tee stream can use a number of strategies to govern how the speed of its outputs affect the backpressure signals it gives off, but the simplest strategy is to pass aggregate backpressure signals directly up the chain, thus letting the speed of the slowest output determine the speed of the tee.” This also differs from analogous operations in other libraries such as repeated nodejs Readable.pipe or akka-streams Source.alsoTo which backpressure to the slower consumer and only require a bounded queue.
As I said in a node-fetch issue https://github.com/node-fetch/node-fetch/issues/1568, I think this unbounded enqueueing algorithm of
ReadableStream.tee()from https://github.com/whatwg/streams/pull/311 was a mistake. The built-in operations of aReadableStreamshould only need a fixed-size queue. , but at this point it is too late to fix it.But at least we can clarify the informational text, which implied that it is safe to
tee()a stream and consume it independently. It is generally not safe to consume atee()’d stream independently unless the difference in consumed data between the faster consumer and the slower consumer fits in memory.(See WHATWG Working Mode: Changes for more details.)
Preview | Diff