cta-wave / dpctf-tests

Repo for DPCTF Tests. We prefer to keep the Tests separated from Test Runner
Other
1 stars 5 forks source link

tests using stream t2 vs tests using chunked content #160

Open jpiesing opened 3 months ago

jpiesing commented 3 months ago

Stream t2 is the version of the chunked content before the chunking script is run on it. 4 generated tests include t2 in the filename.

2 generated tests have chunked in their name

regular-playback-of-chunked-contentchunked regular-playback-of-chunked-content-non-aligned-appendchunked

t2 is not meant to be used without being chunked. Is is correct that the 4 tests are using different content from the 2?

Also note that TVs number 2 and 4 from the London plugfest decode all frames from the chunked tests but drop a lot of frames at the start of the t2 tests.

gitwjr commented 2 months ago

From the DPCTF Call on 2024-04-10 Fraunhofer regenerated using content t1. No difference in P/F or the numbers were found using t1 vs t2. Needs deeper analysis to determine why it is failing. Noted that this test has far fewer failures on a Desktop than on TVs. They will rerun and review the results after changes from issues # 123, 122, 121 and compare the new results with results from testing with a Desktop browser.

jpiesing commented 1 month ago

2024:05-14: On one of the recordings from the London plugfest, this still fails due to a small error in the duration observation. We should wait for Fraunhofer to run the tests on their small zoo and see how common this error is.