Calculate "Average Playback Stalled Count" per time unit

[tonihei]

The current definition of "Average Playback Stalled Count" uses a playback session as the unit for stall count. For example 5 stalls/session. This is extremely dependent on the length of these sessions and makes it difficult to compare player performances to each other as soon as the sessions lengths are not exactly the same.

The metric is potentially more stable and easier to compare if it's defined per time unit, e.g. 5 stalls/hour. The time unit should possibly be "play time", but there are possibly also good arguments to use "watched time" (e.g. it would match the definition of "Average stalled time percentage").

[gheikkila]

This has been heavily discussed in the group, exactly with your arguments. The reason for (so far) keeping the non-normalized "stalls/session" is that many vendors seem to like this, as it's easy to explain for the (upper) management...

[tonihei]

it's easy to explain for the (upper) management.

That doesn't sound like it should be a guiding principle in designing a standard. :)

I suppose the goal of this standard should be to create a minimum list of metrics needed to achieve comparable well-defined playback stats which are useful for all applications. If the only metric for the core QoE concept of playback stalls is one which is not practically useful, people may be reluctant to adopt the standard at all for their QoE efforts. Note how this metric prevents comparisons and doesn't even tell you if something got better or worse (unless all videos happen to be the same length, which is probably not true for most use cases).

[mlevine84]

WG recommends to retain both "average playback stalled count" and "average stalled time percentage" metrics to address the eco-system needs. WG will consider adding the requested time normalized metric.

[mlevine84]

Gunnar agreed to propose text for WGs review.

[mlevine84]

The normalized metric has been added to the spec.