MediaStreamTrack audio delay/glitch capture stats

[henbos]

In the getStats() spec, capture metrics for measuring the capture delay per sample.

Feedback (#741) from @padenot was that WebRTC getStats() is the wrong place to put this.

We need to revert this and have something on the MediaStream that represents an audio input device, or something like this.

It's not desirable to have coupling between the audio input system and WebRTC

Another point was whether this should be per sample (current definition) or per frame (what rest of the platform prefers).

totalSamplesCaptured: this should be named totalFramesCaptured, to handle multichannel and to be consistent with the rest of the Web Platform

In the getStats() case we had chosen samples for consistency with the rest of getStats(), but if we move these metrics to the MediaStreamTrack when we could use frames instead. With the recent addition of track.getFrameStats(), which is basically about capture frame rate, why not add audio capture delay metrics here too?

Proposal: Add capture delay to track.getFrameStats()

[henbos]

We also have the capture glitches which should also be included, you'd want to compute droppedSamplesDuration/totalSamplesDuration

[henbos]

@jan-ivar

[jan-ivar]

Moving them out of peer connection seems like an improvement, since capture metrics seem useful outside of p2p. I.e. users shouldn't need to create an RTCPeerConnection and add a track to it just to get this info.

[henbos]

What WebRTC Stats refers to as audio samples...

An audio sample refers to having a sample in any channel of an audio track - if multiple audio channels are used, metrics based on samples do not increment at a higher rate, simultaneously having samples in multiple channels counts as a single sample.

... and what Mozilla refers to as audio frames ...

While recording or generating multi-channel audio files, the channels are assembled into a series of audio frames, each consisting of one sample for each of the audio's channels.

Appears to be the same thing. Can we make a PR that uses the "audio frames" language but is otherwise more or less the same as what we already have in RTCAudioSourceStats? Specifically interested in:

• totalSamplesDuration
• droppedSamplesDuration
• droppedSamplesEvents
• totalCaptureDelay
• totalSamplesCaptured

@padenot @jan-ivar

[henbos]

Also FYI @vr000m I'm trying to address Mozilla concerns by moving some recently added getStats metrics to the MediaStreamTrack so that capture related stats can be obtained outside of WebRTC.

[henbos]

Friendly ping. Should I provide a PR that moves these metrics here (and changes the language a bit)? @padenot @jan-ivar

[dontcallmedom-bot]

This issue had an associated resolution in WebRTC WG 2023-04-18 – (Issue 93: MediaStreamTrack audio delay/glitch capture stats (Henrik)):

RESOLUTION: Moving them to MediaStreamTrack makes sense, Henrik to write a PR.

[henbos]

This was split up into various issues and PRs, but can now be closed, and the webrtc-stats versions were just removed: https://github.com/w3c/webrtc-stats/pull/773