search

video-dev / standards

Discuss potential standards before creating an initial implementation. Useful for continuing conversations from groups like FOMS.
3 stars 2 forks source link

Media Info API example for bitrate metrics #1

Open heff opened 5 years ago

heff commented 5 years ago

The recent QoE standard in progress requires access to the currently playing bitrate, to calculate bitsPlayed. This isn't easy to get on the web currently. One of the requests at FOMS was something like a Media Info API (similar in concept to the Media Capabilities API) that could provide those details. A request from browser developers was an example of what we would want to see from this API.

So we should create an example of what we think the API could look like. The action item included @robwalch.

A few suggestions from Eric Carlson and others at apple:

Check out WebRTC Stats API for an example of similar stats already made available by the browser. Look to the Intersection Observer API as a better example of an API where you tell the browser what details you want. Otherwise the browser has to dig into the system to get all the info each time which is costly. Look at AV Player HLS stats for example of what people have asked for from that API.

This has been kicked around a few repos looking for a home. Felt like video-dev should have a spot for discussions like this. https://github.com/cta-standards/R4WG20-QoE-Metrics/issues/28#issuecomment-499178030

mlevine84 commented 5 years ago

WG agreed this would be a good subject to be discussed at FOMs: http://www.foms-workshop.org/foms2019SF/.

robwalch commented 5 years ago

The standard player properties in the QoE standard that stand out as not available via HTMLMediaElement are:

I would also recommend a currentTextCodec for subtitle format ("wvtt", "stpp.ttml.im1t").

All of the above may (or should) be available based on a main manifest that describes each rendition in detail. How would you get any of these properties when not using an MSE or WebRTC based video player? (Does AVFramework provide an interface for this info not available in iOS Safari?)

IMO a Media Info API would essentially be the inverse of the Media Capabilities API. Rather than provide a description of media to the browser and ask it if it can play media that matches the description, I would like to be able to query properties of the currently playing media in the same or a similar format to the one which the Media Capabilities API accepts. Examples would be methods, events or an observer which can return objects similar to a VideoConfiguration and/or AudioConfiguration of media that is playing.