low-latency-initialization__t2.html failed on all TV sets in London plugfest

cta-wave / dpctf-tests

Repo for DPCTF Tests. We prefer to keep the Tests separated from Test Runner

Other

2 stars 5 forks source link

low-latency-initialization__t2.html failed on all TV sets in London plugfest #151

Closed jpiesing closed 6 months ago

jpiesing commented 8 months ago

The test low-latency-initialization__t2.html failed the "[OF] Measure the time between the successful appending of the first CMAF chunk that exceeded min_buffer_duration and the first media sample being visible or audible. This value shall be compared against render_threshold" observation in all TV sets in the London plugfest.

jpiesing commented 7 months ago

Excluding dropped frames, most TVs would pass this if the render_threshold was relaxed from its current value of 200ms. render_threshold is included in the DPCTF specification but the threshold is not defined in that document. Somewhere between 650 and 700ms would be sufficient so I propose to relax this to 800ms. This is defined in test-config.json so it's not a code change by Resillion and can be run against the existing recordings.

yanj-github commented 7 months ago

Tested with 800ms on one of the recordings and it passed the test. @FritzHeiden can you change render_threshold=800ms please?

FritzHeiden commented 7 months ago

Parameter updated with https://github.com/cta-wave/dpctf-tests/pull/168

yanj-github commented 7 months ago

low-latency-initialization__t2.html test on 3 TV's [OF] Measure the time between the successful appending of the first CMAF chunk that exceeded min_buffer_duration and the first media sample being visible or audible. This value shall be compared against render_threshold.:

PASS: Maximum permitted rendering delay is render_threshold=800ms.The presentation rendering delay is 604.1662ms
PASS: Maximum permitted rendering delay is render_threshold=800ms.The presentation rendering delay is 331.589ms
PASS: Maximum permitted rendering delay is render_threshold=800ms.The presentation rendering delay is 326.3333ms

Changes confirmed. @jpiesing please close the issue when you happy with the result above.

jpiesing commented 7 months ago

Changes confirmed. @jpiesing please close the issue when you happy with the result above.

It cannot be closed yet due to the missing frames failure - https://github.com/cta-wave/device-playback-task-force/issues/126 .

FAIL: First frame found is 4, expected to start from 1. First frame number tolerance is 0. Mid frame number tolerance is 10. Total of missing frames is 3.

jpiesing commented 6 months ago

Since we are considering https://github.com/cta-wave/device-playback-task-force/issues/126 to be a TV implementation error then I think we can close this.