search

w3c / mediacapture-extensions

Extensions to Media Capture and Streams by the WebRTC Working Group
https://w3c.github.io/mediacapture-extensions/
Other
19 stars 14 forks source link

Clarifying Transferable MediaStreamTrack Types #154

Open bc-lee opened 2 weeks ago

bc-lee commented 2 weeks ago

It's promising to see support for transferring MediaStreamTrack to Web Workers in the latest Safari Technology Preview 196 and Safari Beta 18. I successfully built a demo that utilizes transferred MediaStreamTrack as input for MediaStreamTrackProcessor and VideoTrackGenerator.

However, Chromium Canary's implementation under the MediaStreamTrackTransfer flag seems to have limitations. Attempting to transfer a MediaStreamTrack obtained from getUserMedia (gUM) within the window context to a Web Worker results in this error:

DataCloneError: Failed to execute 'postMessage' on 'Worker': MediaStreamTrack could not be serialized.

This suggests that certain MediaStreamTrack types might not be transferable due to various reasons. Unfortunately, I couldn't find any definitive information about transferable MediaStreamTrack types until diving into Chromium's source code (specifically, the IsMediaStreamDeviceTransferrable function within media_stream_request.cc).

Establishing a clear consensus on which MediaStreamTrack types are transferable, followed by documentation within the specification, would be beneficial for web developers.

youennf commented 2 weeks ago

I think all MediaStreamTrack(s) are expected to be transferable (except CanvasCaptureMediaStreamTrack since it is not exposed in DedicatedWorker).

AFAIK, Chrome is not implementing yet MediaStreamTrack transfer to DedicatedWorker. See for instance https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/renderer/modules/mediastream/media_stream_track.idl. Filing a bugs on chromium bug tracker might be the best approach.