Open aboba opened 1 year ago
Dom & Francois have demonstrated a way of tracking timing at each stage of the processing pipeline, by inserting a timestamp overlay.
It would be nice to have a consistent timing model built in so as to make this more straightforward.
The requestVideoFrameCallback specification defines VideoFrameCallbackMetadata, which includes some timing information.
It has been proposed that some fields from VideoFrameCallbackMetadata be added to VideoFrameMetadata: https://github.com/w3c/webcodecs/issues/601
VideoFrameCallbackMetadata
VideoFrameMetadata
However, currently timing metadata is not consistently exposed across media processing APIs. For example:
VideoFrame.captureTime
VideoFrame
MediaStreamTrack
Related slides: https://docs.google.com/presentation/d/1DEk4urMPNufIYX88jzRqXddxoGRZXv10iOTGbKvLe5s/edit#slide=id.g159ea124a3f_8_221
Dom & Francois have demonstrated a way of tracking timing at each stage of the processing pipeline, by inserting a timestamp overlay.
It would be nice to have a consistent timing model built in so as to make this more straightforward.
The requestVideoFrameCallback specification defines VideoFrameCallbackMetadata, which includes some timing information.
It has been proposed that some fields from
VideoFrameCallbackMetadatabe added toVideoFrameMetadata: https://github.com/w3c/webcodecs/issues/601However, currently timing metadata is not consistently exposed across media processing APIs. For example:
VideoFrame.captureTimeis not exposed within the mediacapture-transform API (e.g. in aVideoFrameproduced from aMediaStreamTrack)VideoFrameis passed to WebCodecs Encoder, the timing metadata is not copied to the encoded ChunkRelated slides: https://docs.google.com/presentation/d/1DEk4urMPNufIYX88jzRqXddxoGRZXv10iOTGbKvLe5s/edit#slide=id.g159ea124a3f_8_221