missing frames at start of many tests on London plugfest TV #2 and #4

[jpiesing]

TV 2 and 4 are the two best TVs in the London plugfest. Both of them fail a number of tests due to missing frames at the start. This applies particularly to the second batch of tests.

Test	TV 2	TV 4
buffer-underrun-and-recovery__t2	FAIL: First frame found is 6, expected to start from 1. First frame number tolerance is 0. Mid frame number tolerance is 10. Total of missing frames is 5.	PASS
low-latency-initialization__t2	FAIL: First frame found is 4, expected to start from 1. First frame number tolerance is 0. Mid frame number tolerance is 10. Total of missing frames is 3.	FAIL: First frame found is 4, expected to start from 1. First frame number tolerance is 0. Mid frame number tolerance is 10. Total of missing frames is 3.
low-latency-playback-over-gaps__t2	FAIL: First frame found is 7, expected to start from 1. First frame number tolerance is 0. Mid frame number tolerance is 10. Frames out of order 368, 251. Last frame detected before gap 120 is within the tolerance of 'stall_tolerance_margin'=7.5 frames of expected frame 125. Total of missing frames is 6.	FAIL: First frame found is 4, expected to start from 1. First frame number tolerance is 0. ......
low-latency-short-buffer-playback__t2	FAIL: First frame found is 6, expected to start from 1. First frame number tolerance is 0. Mid frame number tolerance is 10. Total of missing frames is 5.rst frame found is 6, expected to start from 1. First frame number tolerance is 0. Mid frame number tolerance is 10. Total of missing frames is 5.	FAIL: First frame found is 6, expected to start from 1. First frame number tolerance is 0. Mid frame number tolerance is 10. Total of missing frames is 5.
random-access-from-one-place-in-a-stream-to-a-different-place-in-the-same-stream__t2	FAIL: First frame found is 7, expected to start from 1. First frame number tolerance is 0. ....	FAIL: First frame found is 5, expected to start from 1. First frame number tolerance is 0. .....
playback-of-encrypted-content-https__t1-cenc	FAIL: First frame found is 4, expected to start from 1. First frame number tolerance is 0. Mid frame number tolerance is 10. Total of missing frames is 3.	PASS

The cause of this is unclear. It is doubly unclear why this mostly happens with the 2nd batch of tests and only once with the first batch of tests.

@yanj-github has noted that some TV sets fade in the video and the QR codes may not be detected during this. Someone needs to look at the recordings for TV 2 and TV 4 to see if this is the case or if the frames concerned are actually missing.

Either way, someone needs to review the code between the 1st and 2nd batch to see if the sequence of API calls is the same or different.

[jpiesing]

FritzHeiden Please can you review the last point - compare the sequence of API calls between the 1st batch of tests and the 2nd to see if there's a difference in what API calls are made and the order they're made in.

[yanj-github]

I have checked recording on some of these tests. Missing frames are correctly reported by OF. A short period of TS stream plays between pre test QR code and start playing the actual test content. I am not sure this could be an impact to start frames missing?

[jpiesing]

I have checked recording on some of these tests. Missing frames are correctly reported by OF. A short period of TS stream plays between pre test QR code and start playing the actual test content. I am not sure this could be an impact to start frames missing?

@FritzHeiden I cannot see anyone other than you who can investigate the behaviour reported by Yan above.

[jpiesing]

@fritzheiden @louaybassbouss You have probably thought of most of these but, just in case, here are my thoughts on experiments that might help.

1. In case the problem is the TV.

   • Run the tests in a different order (to the extent the test runner can do this)
   • Run the same test several times (to the extent the test runner can do this)
   • Run the problematic tests directly after a power cycle of the TV
   • If your current HbbTV wrapper uses the release() method on a v/b object then try calling setChannel(null) and vice-versa.

2. In case the problem is the test content.

   • Run all the sequential track playback tests with the alternate content that is not normally used, t10 to t15, t21 to t34, see if any of them show the error

3. If the problem is the HTML+JS code.

   • See if any of the HTML+JS templates which fail can be run with with stream t1 and run them with that

[FritzHeiden]

What we discovered was that first frame are not dropped after a full power cycle, however, consecutive test runs had dropped frames again. The logic for player initialization does not differ from tests that are already considered to be validated, like sequential-track-playback. Right now we are still looking into the hbbtv wrapper logic.

[louaybassbouss]

Update: We ran also tests in TV Browser (not HbbTV) and we got there also skipped frames at the beginning (similar behaviour between HbbTV and TV Browser).

[haudiobe]

DPCTF:

• TV implementation bug
• won't fix unless we get new evidence

[jpiesing]

Can we close this?