w3c / webrtc-encoded-transform

WebRTC Encoded Transform
https://w3c.github.io/webrtc-encoded-transform/
Other
123 stars 27 forks source link

Clarify why the writable stream backpressure should be disabled #188

Closed saschanaz closed 1 year ago

saschanaz commented 1 year ago

https://w3c.github.io/webrtc-encoded-transform/#stream-creation

Set up this.[[writable]] with its writeAlgorithm set to writeEncodedData given this as parameter and its sizeAlgorithm to an algorithm that returns 0.

Chunk size is set to 0 to explictly disable streams backpressure on the write side.

https://github.com/w3c/webrtc-encoded-transform/pull/108 disabled it and added some notes, but the intention is not clear why it should be disabled. The note only says it's disables by letting sizeAlgorithm to return 0 (which I think it should set highWaterMark to Infinity instead).

youennf commented 1 year ago

I think it should set highWaterMark to Infinity instead

We can do this.

As of why it is disabled, the principle is that the pipeline is a realtime pipeline where buffering should be very limited. Backpressure should be handled either at input or at output by the UA. For instance, if the writer is pushing data to the network, packets might be dropped if overshooting network, or the encoder will reduce the size of next frames to limit throughput. In any case, backpressure should not be used to let the web applications delay its transform processing.

alvestrand commented 1 year ago

This is the wrong direction to go in. Depending on the UA knowing how the input of the transform connects to the output of the transform is the wrong model, and needs to be abandoned; for one thing, it does not permit one-ended use cases.

The right way of approaching this is to make the feedback channel explicit as outlined in https://github.com/alvestrand/hackathon-encoded-media

EDIT: It seems that we have already disabled backpressure in the specified API, so we have no defense against buffer overruns. This seems dangerous, but it is a previously accepted danger.

dontcallmedom-bot commented 1 year ago

This issue was discussed in WebRTC July 2023 meeting – (Issue #188: Clarify why backpressure should be disabled)