can not get video frame's dts time stamp

[shaozj]

In the audio-video-player sample, we pass EncodedVideoChunk a cts time stamp for encode video chunk。However，we can not get dts time stamp from the encoded video frame. which makes error when we seek a frame use time stamp，because the accurate display time stamp is dts. How can I fix this problem?

[dalecurtis]

I'm confused, AFAIK composition timestamps (CTS), presentation timestamp (PTS), and display timestamps (not normally called DTS) are always the same. DTS is typically used to refer to "decode timestamp" which is only applicable in out of order encoding situations -- for which I don't know of a current implementation having such support.

You should always seek using the presentation timestamp and the structure you use to seek should be aware that seeking in out-of-order content may require some additional metadata/lookup to complete.

We probably do need to specify what happens with timestamps in B-frame encoding, but I defer to @sandersdan for the plan there.

Can you elaborate on your issue some more?

[shaozj]

I found the inaccurate seek is caused by video track's media_time，which seems same to the first frame's cts. When I calculate the timestamp, I minus media_time, then I get the accurate time stamp. I didn't find pts in track info parsed by mp4box.js

[shaozj]

[sandersdan]

It is true that WebCodecs does not emit a DTS with EncodedVideoChunks, however it is straightforward to generate these. Simply start at 0 for the first frame, and then in decode order (which is the order encoded chunks are emitted) add each frame duration.

There may be containers where it is required that DTS not exceed PTS, which could be difficult to precisely accomplish with WebCodecs. This isn't currently an issue with Chrome's implementation since we never emit H.264 B-frames, but if/when it is possible it can usually be worked around by adding a safe offset to the PTS (and a matching CTS edit).