search

w3c / mediacapture-handle

https://w3c.github.io/mediacapture-handle/
Other
14 stars 10 forks source link

.sendCaptureAction() seems misplaced on the track #65

Open jan-ivar opened 2 years ago

jan-ivar commented 2 years ago

track.sendCaptureAction, the API to send supported actions to the captured page seems misplaced on the MediaStreamTrack. The reasons are:

  1. MediaStreamTrack is a clonable & constrainable media consumption abstraction. One source → many consumers. Whereas methods like track.cropTo are per-clone, sendCaptureAction would affect ALL clones, a leaky abstraction. The progression of the capture is not a property of a single video track.
  2. Would both the video and audio track have this method? Why (not)? It's confusing.

We’ve stepped out of media consumption to remote control.

A higher level object seems needed, one that isn't clonable or necessarily shared with every media consumer. Likewise, https://github.com/w3c/mediacapture-screen-share/issues/190 seems to similarly need this higher level object.

See https://github.com/w3c/mediacapture-handle/issues/12#issuecomment-1051021052 for ideas on such a higher level object, which I'm scheduled to present tomorrow.

eladalon1983 commented 2 years ago

sendCaptureAction would affect ALL clones, a leaky abstraction.

It does not affect all clones, or any of the clones. The tracks themselves are not affected by sendCaptureAction. The tracks are merely conduits. All roads lead to Rome - the single captured tab, and its single registered event handler.

Would both the video and audio track have this method?

The same reasoning (all roads lead to Rome) convinces me that it would be fine to expose on both audio and video.

youennf commented 2 years ago

My general mental model is that a method that only impacts the track should go to the track. A method that impacts the source (so all tracks) is better to keep outside of tracks.

Having a single place for actions is a good simplification and will make things simpler. Having the ability to transfer a track but keep control of the capture/action manager adds flexibility to web applications, without making them any less powerful.

It also seems like it will be a simpler and less error prone programmatic model for web developers. It might also be easier to implement for UA, past the boiler plate to add the probably new interface required by such a container.

Overall adding this separate container looks like a nice addition, so it seems good to have a try. We should probably dive into the details to fully validate this. For instance, making it Transferable like MediaStreamTrack might be a good idea as some applications might want to delegate both track and action handling to a separate context (say a worker). But this might have impact on the corresponding API the capturer might support.

alvestrand commented 2 years ago

If we want to think about exposing the source as an object, we'd better think about exposing the source as an object.

Some sources (VTG, capture-from-element) are objects already, but there's no generic source object.