search

cta-wave / device-observation-framework

This repository contains the WAVE Device Observation Framework. The DPCTF Device Observation Framework determines pass or fail results, based on observations taken of tests which are run on a device by the DPCTF Test Runner. DPCTF is the Device Playback Compatibility Task Force of the CTA WAVE Project (https://CTA.tech/WAVE and standards@CTA.tech for more info).
Other
1 stars 3 forks source link

Does issue in #48 result in failure of playback duration sub-test? #49

Closed jpiesing closed 1 year ago

jpiesing commented 1 year ago

The discussion in #48 explains an issue where, if video frames are slow to appear, the errors are generated when playback has theoretically started by currentTime has not yet started advancing. For the playback duration sub-test, would the same issue apply? When is the playback duration calculated from? If there would be 4-5 frames where currentTime was 0.0 but no video had appeared, would this result in the duration as calculated by the OF being increased by 4-5 frame periods and hence the playback duration sub-test failing?

yanj-github commented 1 year ago

Each observation is separately made and failing one should not cause fail to others. The current Time match check is not related to the duration check.

actual_playback_duration should match expected_track_duration within +/- tolerance. Where the actual_playback_duration is calculated from recording time of 1st detected video frame till the recording time of last detected video frame. And expected_track_duration is being adjusted based on the missing frames either end, start and end.

If there would be 4-5 frames where currentTime was 0.0 but no video had appeared, would this result in the duration as calculated by the OF being increased by 4-5 frame periods and hence the playback duration sub-test failing?

This is not the case. If there are 4 frames missing at the begining, the actual_playback_duration will be calculated from frame number 5 till the last frame. And expected_track_duration will be adjusted by minus 4 frame worth the duration.

For more information of detailed algorithm can be found here: https://github.com/cta-wave/device-observation-framework/wiki/Observation-Algorithms#the-playback-duration-of-the-playback-matches-expected-duration

jpiesing commented 1 year ago

Closing after further analysis of the failing OF tests.