Open henbos opened 9 months ago
For background, the last time this API was discussed in the WebRTC Working Group was at TPAC, where we decided to make the API a synchronous getter instead of a promise-based API that resolves with the result: recording with timestamp, slides
Chrome has already shipped video track stats, I am planning to do an Intent to Ship for the audio track stats as well. Shall I reuse this "standards-position" to cover both audio and video track stats or should I file a new one for the audio track stats case, @youennf ?
The update is that we've extended this API to also include getUserMedia
-sourced audio tracks:
https://w3c.github.io/mediacapture-extensions/#the-mediastreamtrackaudiostats-interface
The audio metrics defined include:
track.stats.latency
measures the microphone input delay. We also have a PR that adds min/max/average latency since resetLatency()
is called which has not merged at the time of writing but it should be included in the spec since it was approved by the working group in the meeting yesterday.
WebKittens
@youennf @jyavenard
Title of the spec
Media Capture and Streams Extensions, section "MediaStreamTrack Statistics"
URL to the spec
https://w3c.github.io/mediacapture-extensions/#mediastreamtrack-statistics
URL to the spec's repository
https://github.com/w3c/mediacapture-extensions
Issue Tracker URL
No response
Explainer URL
No response
TAG Design Review URL
No response
Mozilla standards-positions issue URL
https://github.com/mozilla/standards-positions/issues/895
WebKit Bugzilla URL
No response
Radar URL
No response
Description
Video conferencing web developers (Google Meet) have asked for an API to know the actual camera frame rate as well as an ability to get a signal from the stack if frames are being dropped somewhere in the pipeline.
The problem with
MediaStreamTrack.getSettings().frameRate
is that it only provides you with the camera's configured frame rate, say 30. In reality, it is common for the actual frame rate to vary such as due to low lighting conditions. In rare circumstances, frames could even get dropped by the user agent, such as if the system is under heavy load or buffers are not being released on time for some reason.The track stats API provides valuable insights. Here are real world scenarios that have been observed, but have been hard to investigate without this API:
track.stats.totalFrames
/s) in half.++track.stats.discardedFrames
) and needs to send to produce another frame (else the next presentation slide is never encoded and sent to the other participants).track.stats.totalFrames - track.stats.deliveredFrames - track.stats.discardedFrames > 0
). This is rare but it deserves to be flagged when it happens.Having an API can help notify the user, troubleshoot (e.g. USB hub...) or provide A/B diagnostics when a web app is launching new features. The motivation is similar to the
RTCPeerConnection.getStats()
API, but there areMediaStreamTrack
use cases that don't depend on WebRTC so it makes more sense to have the API on the track.The metrics are expressed as total counters as opposed to frame rates because this allows the app to calculate average frame rates over any interval (by polling twice and looking at delta frames / delta seconds) or to provide total number of frames dropped or discarded for the entire capture session. Some apps want rates for every second, others every 10 seconds, and other may just care about the session total. In the case of discarded/dropped frames there are use cases where total counters are preferred, or else the app has to poll excessively in order not to "miss" any time a drop happened.
Other information
At the time of writing, the track.stats API is only defined for video tracks. I plan to submit a PR that also adds audio stats which will have different metrics definition and a different interface, but the API shape will follow the same pattern, i.e. the app should be able to call track.stats on an audio track in the future as well.
Edit audio track stats are now also part of the spec, see comments below