search

w3c / mediacapture-transform

MediaStreamTrack Insertable Media Processing using Streams
https://w3c.github.io/mediacapture-transform/
Other
44 stars 20 forks source link

What is the timestamp value of the VideoFrame/AudioData from a remote track? #87

Open murillo128 opened 2 years ago

murillo128 commented 2 years ago

I have not been able to find if there is any specification about what is the value of the timestamp attribute of an VideoFrame or AudioFrame from a remote MediaStreamTrack recevied by a webrtc peerconnection.

Is is specified anywhere?

alvestrand commented 2 years ago

Interestingly, this has come up in other contexts today. I think we need to find an answer (preferably a consistent one). Have you observed this value in the current state of implementation?

murillo128 commented 2 years ago

It seems that it is using the elapsed time since the first received frame (i.e. relative timestamp since the beginning of the call, starting at 0)

https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/renderer/platform/webrtc/convert_to_webrtc_video_frame_buffer.cc;drc=047c7dc4ee1ce908d7fea38ca063fa2f80f92c77;l=347

https://jsfiddle.net/ytqn98eb/

youennf commented 2 months ago

This is being discussed in https://github.com/w3c/mediacapture-transform/issues/96. It seems good to me for each spec defining a track source to define how to extract a VideoFrame from it.

As of webrtc video frames, the closest I could get is the definition of captureTime: For video frames coming from remote source, the capture time is based on the RTP timestamp of the frame and estimated using clock synchronization. This is best effort and can use methods like using RTCP SR as specified in RFC 3550 Section 6.4.1, or by other alternative means if use by RTCP SR isn’t feasible.

dontcallmedom-bot commented 2 months ago

This issue was discussed in Joint Media/WebRTC WG meeting at TPAC – 26 September 2024 (What is the timestamp value of the VideoFrame/AudioData from a remote track?)