search

cta-wave / device-observation-framework

This repository contains the WAVE Device Observation Framework. The DPCTF Device Observation Framework determines pass or fail results, based on observations taken of tests which are run on a device by the DPCTF Test Runner. DPCTF is the Device Playback Compatibility Task Force of the CTA WAVE Project (https://CTA.tech/WAVE and standards@CTA.tech for more info).
Other
1 stars 3 forks source link

random_access_from_tolerance error message wrong #76

Closed jpiesing closed 6 months ago

jpiesing commented 7 months ago

The same recording that's in https://github.com/cta-wave/device-observation-framework/issues/75 is also failing as follows;

    {
        "status": "FAIL",
        "message": " Mid frame number tolerance is 10. Last frame detected before gap 130 is within the tolerance of 'random_access_from_tolerance':5.0 frame of expected frame 125. Samples within a gap shall not be rendered. 1 number of following frames should not be rendered. 131 Total of missing frames is 0.",
        "name": "[OF] Every video frame S[k,s] shall be rendered and the video frames shall be rendered in increasing presentation time order."
    },

Frame 131 is in fact the last frame detected before the gap not 130.

yanj-github commented 7 months ago

Thanks reporting @jpiesing this is confusing, we will change the message. Code change but trivial.

jpiesing commented 7 months ago

See also https://github.com/cta-wave/dpctf-tests/issues/167 .

jpiesing commented 7 months ago

This issue impacts multiple TVs from the London plugfest. image

yanj-github commented 7 months ago

@jpiesing changes made and we are reviewing it now, it reports: _Mid frame number tolerance is 10. Following frames are missing: 3 7 17 Last frame detected before gap 137 exceeded 'random_access_fromtolerance':5.0 frame of expected frame 125. Total of missing frames is 3. Feel free to let me know if you have further comments, otherwise, I will commit the changes later.

jpiesing commented 7 months ago

@jpiesing changes made and we are reviewing it now, it reports: _Mid frame number tolerance is 10. Following frames are missing: 3 7 17 Last frame detected before gap 137 exceeded 'random_access_fromtolerance':5.0 frame of expected frame 125. Total of missing frames is 3. Feel free to let me know if you have further comments, otherwise, I will commit the changes later.

Is it practical to change the order of the items in the message, e.g. to move the 'Total of missing frames' before 'Last frame detected before gap'? Otherwise fine.

yanj-github commented 7 months ago

Fix is now committed the main.

yanj-github commented 6 months ago

FAIL: First frame found is 5, expected to start from 1. First frame number tolerance is 0. Mid frame number tolerance is 10. Total of missing frame count is 4. Last frame detected before gap 131 is within the tolerance of 'random_access_from_tolerance':10.0 frame of expected frame 125. First frame detected after gap 376 doesn't matches expected frame 377.

PASS: Mid frame number tolerance is 10. Total of missing frame count is 0. Last frame detected before gap 131 is within the tolerance of 'random_access_from_tolerance':10.0 frame of expected frame 125.

@jpiesing I am sharing reported result messages. Can this issue be closed now please?