search

w3c / mediacapture-transform

MediaStreamTrack Insertable Media Processing using Streams
https://w3c.github.io/mediacapture-transform/
Other
44 stars 19 forks source link

Access to raw metadata #3

Closed youennf closed 3 years ago

youennf commented 3 years ago

I can see use cases where it would be useful to have metadata attached to raw frames be exposed to JavaScript. This could for instance be head detection metadata that some cameras already generate. Or it could be a track processor that adds such metadata in addition to (or in lieu of) modifying the pixels.

There may be also use cases where this metadata should be preserved by the encoder (or decoder) so that the JS raw frame metadata would be kept attached to the JS encoded frame (or vice versa).

alvestrand commented 3 years ago

Absolutely agree. But given that we are adopting the frames from WebCodecs, can you raise an issue on this in the WebCodecs repo and link it here?

alvestrand commented 3 years ago

@youennf ?

youennf commented 3 years ago

Thanks for the ping, will do it shortly.

alvestrand commented 3 years ago

Great, closing in this repo.

youennf commented 3 years ago

I think there are WebRTC bits we need to discuss outside of WebCodecs. In particular, we want that metadata added by media capture raw goes up to WebRTC encoded transform, which is not WebCodec.

Reopening this issue for now. Though maybe we could move it to webrtc encoded transform.

guidou commented 3 years ago

I think we have been discussing this in the context of encoded transform, so closing this.