search

cta-wave / dpctf-tests

Repo for DPCTF Tests. We prefer to keep the Tests separated from Test Runner
Other
1 stars 5 forks source link

fullscreen-playback-of-switching-sets__ss1-1.html failed in all TV sets in London plugfest #150

Open jpiesing opened 3 months ago

jpiesing commented 3 months ago

In both fullscreen-playback-of-switching-sets__ss1-1.html and ss1-2, the observation "[OF] Video: The presented sample shall match the one reported by the currentTime value within the tolerance of +/-(2/framerate + 20ms)" failed on all 8 TV sets in the London plugfest.

yanj-github commented 2 months ago

Hi @jpiesing, After change the tolerance to 1frame + 150ms now just failing on one occasion. It is the point when switching happens. We did double check the OF to make sure it uses right frame rate. _Allowed tolerance is 1 frames, 150ms. Time difference between Test Runner reported media currentTime and actual media time exceeded tolerance for following events: currentTime=46.11 timediff=210.0; Total failure count is 1.

@FritzHeiden can you kindly help to double check the currentTime report at the switching point to make sure it reported correctly please?

FritzHeiden commented 2 months ago

@yanj-github Updating the qr code is coupled to several events. Some of them include: video enters waiting state, currently playing representation changes, video buffers, video starts playing, timeupdate event

Anytime the qr code is updated it uses the currentTime value reported by the API

jpiesing commented 2 months ago

@yanj-github Updating the qr code is coupled to several events. Some of them include: video enters waiting state, currently playing representation changes, video buffers, video starts playing, timeupdate event

Anytime the qr code is updated it uses the currentTime value reported by the API

@FritzHeiden @yanj-github Would it help if the two of you discussed this in a zoom/teams call? Otherwise I'm not sure this is going anywhere.

yanj-github commented 2 months ago

@jpiesing and @FritzHeiden I have created a new issue for this https://github.com/cta-wave/dpctf-tests/issues/172

jpiesing commented 2 months ago

@yanj-github Now #172 is closed, is there anything left in this issue?

yanj-github commented 2 months ago

@jpiesing It can be closed if you are happy. The latest result I sent you these are all PASS now.

jpiesing commented 1 month ago

@jpiesing It can be closed if you are happy. The latest result I sent you these are all PASS now.

That's not correct - at least how I interpret the words. Of course you may have meant something different.

I can see failures due to last frame not being shown - which may be another example of https://github.com/cta-wave/dpctf-tests/issues/156.

ss1-2 fails with a duration error.

yanj-github commented 1 month ago

@jpiesing I thought this issue only meant current time match test. If you meant test with other observations then yes some observations failed.

Regarding to duration check it might be related to https://github.com/cta-wave/device-observation-framework/issues/59#issuecomment-2109624659. A1 (low resolution) presented at the beginning this is harder to detect when fading in it might impact how accurate we can calculate duration. Here are the options:

jpiesing commented 1 month ago

2024-05-28: This remains open until it's been run with the new debugging support so we can see what happens with the last frame. It needs to be re-run on the exact same model of TV as exhibited the original problem.