dropped frames from middle may pass threshold but fail on duration

[jpiesing]

There is a threshold of 10 for dropped frames in the middle of a sequence for the observation

"[OF] Every video frame S[k,s] shall be rendered and the video frames shall be rendered in increasing presentation time order."

Some implementations drop frames and do not preserve the media timeline resulting in failure of the duration observation.

If it's possible, a failure of the duration observation when 'every video frame' also fails should be resulted as something other than 'FAIL', particularly if the duration would pass if adjusted for the number of frames dropped.

[yanj-github]

IF I remember correctly, 10 missing frame tolerance was introduced to overcome false negatives due to recording not clear enough to detect the every frames. Either cause by the recording device not robust enough or the playback screen is too small (on the phone) with low resolution content. I think we need more thinking on this. But my 1st impression is that when we having new observation https://github.com/cta-wave/device-playback-task-force/issues/119 added this will be clearer.

[yanj-github]

@FritzHeiden and @louaybassbouss does test runner accept any other result status than "PASS" or "FAIL"? If this is purely a string can I report "WARNING"?

[jpiesing]

IF I remember correctly, 10 missing frame tolerance was introduced to overcome false negatives due to recording not clear enough to detect the every frames. Either cause by the recording device not robust enough or the playback screen is too small (on the phone) with low resolution content. I think we need more thinking on this. But my 1st impression is that when we having new observation cta-wave/device-playback-task-force#119 added this will be clearer.

From recordings I've seen, there are genuine missing frames and not just false negatives.

[yanj-github]

Regarding to every frame rendered: From spec is says "Every video frame S[k,s] shall be rendered" it sound to me that we should not allow any mid missing frames. What do you think @jpiesing?

For the duration check: we can pass the observation if adjusted for the number of frames dropped in the middle. Which then require spec wording to be changed as follows: The playback duration shall match the duration of the CMAF Track – i.e., TR[k,S] = TR[k,1] + td[k] taking into account: • Missing frames, • Potential start frames being rendered before playback, • Frozen last frame after the playback ended.

[jpiesing]

There are two possible variations on missing frames.

• Missing frames where the media timeline is preserved, i.e. the following frames are shown at the correct time, which means the frame before the missing one is prolonged
• Missing frames where the media timeline is not preserved and a frame duration is just dropped from the media timeline.

The first of these is significantly less wrong than the second, particularly for live.

[yanj-github]

Missing frames where the media timeline is not preserved and a frame duration is just dropped from the media timeline.

We will change the OF to make the duration observation also PASS with additional explanation in message.

[jpiesing]

Missing frames where the media timeline is not preserved and a frame duration is just dropped from the media timeline.

We will change the OF to make the duration observation also PASS with additional explanation in message.

For which of the variations in my earlier comment or both?

[yanj-github]

Missing frames where the media timeline is not preserved and a frame duration is just dropped from the media timeline.

We will change the OF to make the duration observation also PASS with additional explanation in message.

For which of the variations in my earlier comment or both?

• Missing frames where the media timeline is preserved, i.e. the following frames are shown at the correct time, which means the frame before the missing one is prolonged

This is already a PASS from OF at the moment.

• Missing frames where the media timeline is not preserved and a frame duration is just dropped from the media timeline.

We will PASS this as well and add additional message in result. e.g: "status": "PASS", "message": "Playback duration is 59733.33ms, expected track duration is 59980.0ms. Detected duration is different by 246.67ms. The media timeline is not preserved for mid missing frames and 200ms duration is dropped from the media timeline. Allowed tolerance is 50ms and duration frame tolerance is 0. Starting missing frame number is 0. Ending missing frame number is 1.", "name": "[OF] Video: The playback duration shall match the duration of the CMAF Track"

[yanj-github]

@jpiesing I think update to spec is required.

[jpiesing]

@jpiesing I think update to spec is required.

Perhaps adding something to 8.2.5.2 like shown in italics?

If the track is a video track, then the following additional observations are expected: 1) Every video frame S[k,s] shall be rendered such that it fills the entire video output window following the properties in clause 5.2.2. 2) The presented sample shall match the one reported by the currentTime value within the tolerance of +/- (2/framerate + 20ms). 3) Every video frame S[k,s] shall be rendered and the video frames shall be rendered in increasing presentation time order. 4) The playback duration shall additionally take into account video frames S[k,s] that are not rendered with the same tolerance as the previous observation. 4) Each video frame S[k,s] with s=1, …, S shall be

There may be some other places as well as 8.2.5.2 where it would need to be added.

[yanj-github]

Hi @jpiesing, Can we not just change "Missing starting and ending frames" to "missing frames"?

I think it is easier to change OF and maintain, and simpler to change the spec not to take tolerance 10 account in duration calculation.

e.g: If 11 missing frames it fails every frames rendered check (more then 10 is missing) but might pass the duration for "Missing 11 frames where the media timeline is not preserved and a frame duration is just dropped from the media timeline." Let me know if you prefer link this with mid missing frame tolerance.

The playback duration shall match the duration of the CMAF Track – i.e., TR[k,S] = TR[k,1] + td[k] taking into account: • Missing frames, • Potential start frames being rendered before playback, • Frozen last frame after the playback ended.

It would be easier to do replace in word document.

[jpiesing]

It would be easier to do replace in word document.

I agree that it would be a lot easier. I've created a new issue in the dpctf repo.

[yanj-github]

Changes now committed to main.

[jpiesing]

Changes now committed to main.

Do you believe this can be closed when https://github.com/cta-wave/device-playback-task-force/issues/127 is approved in DPCTF?

[yanj-github]

@jpiesing yes, if we agree to change "Missing starting and ending frames" with "Missing frames".

[yanj-github]

@jpiesing This issue can be closed now, feel free to close this issue when you happy with the change please.