Open aboba opened 1 year ago
Should the VideoFrame
's timestamp relate to this as well? The webcodecs spec defines it as "The presentation timestamp", with a note about potentially using it to guide rate control in the encoder.
It would be great if we could specify its behaviour in the context of mediacapture transforms.
I think the timestamps in the stream coming from VTG are the timestamps coming into the VTG - dejittering should happen just before playback (or other process that needs a regular cadence).
Needs discussion.
VideoTrackGenerator
(the successor to MSTG) converts a stream ofVideoFrame
s to aMediaStreamTrack
.VideoFrame
s may have been "transformed" by a process that generates jitter. For example, they may have been processed by a Machine Learning algorithm that takes a varying amount of time, or they may have been produced by a decoder that received encoded chunks over a network with varying delays.Currently, the specification does not talk about what an application can expect of
VideoTrackGenerator
in terms of playout delay and rendering smoothness. DoesVideoTrackGenerator
assume that there is a jitter buffer upstream that generatesVideoFrames
with constant delays? Or doesVideoTrackGenerator
adjust the playout delay in response to the jitter it experiences in theVideoFrames
it receives?