search

w3c / webcodecs

WebCodecs is a flexible web API for encoding and decoding audio and video.
https://w3c.github.io/webcodecs/
Other
939 stars 132 forks source link

Need EncodedChunk to have video metadata to support WebRTC #245

Open alvestrand opened 3 years ago

alvestrand commented 3 years ago

The WebRTC Encoded Transform spec's VideoFrame object (https://w3c.github.io/webrtc-encoded-transform/#rtcencodedvideoframe) has a getMetadata function that returns information about the frame that is produced by the encoder, and needs to be passed on to consumers (and vice versa for decoders).

If we're going to harmonize RTCEncodedVideoFrame with EncodedVideoChunk, we need some way to provide this functionality.

(This bug is created as part of bug triage on webrtc-encoded-transform; all bugs that are about buffer formats should move here, I think.)

aboba commented 3 years ago

@alvestrand MetaData Proposal: https://docs.google.com/document/d/1O6kxAQPLWhCRHkH-6zutcqEUOUWRS6NFM2CQAvi1VTg/

Do you need anything beyond that?

aboba commented 3 years ago

@alvestrand Looks like we need to plan for metadata extensibility, so moving the SVC metadata into its own svc dictionary probably makes sense.

aboba commented 2 months ago

@alvestrand @Djuffin @padenot Do we still believe that changes to EncodedChunk are needed to support WebRTC? Or is the required metadata being added to RTCEncoded{Audio, Video}Frame instead?

padenot commented 1 month ago

For WebRTC folks to answer.