search

Dash-Industry-Forum / DASH-IF-Conformance

This repository provides the source code for MPEG-DASH/DASH-IF Conformance Software/Validator. It has been extended according to further standards, such as CMAF, DVB-DASH, HbbTV, and CTA WAVE.
https://conformance.dashif.org/
Other
68 stars 28 forks source link

Dash Validator v1.1 - did something break? #218

Closed richardbushell closed 6 years ago

richardbushell commented 8 years ago

We have over 100 videos online that all previously fully passed the ONLINE Dash Validator previously.

Now, they all throw an error.

Here's an example MPD:- https://living.video/MPD/test6.mpd

Each video representation reports the following error:-

5130 non-sync samples found out of total 5184 samples, for non-indexed track 1, violating Section 6.2.3.2. of ISO/IEC 23009-1:2012(E): every access unit of the non-indexed streams shall be a SAP of type 1.

Really weird! All of these streams previously passed perfectly and nothing has been changed. They are all Segment Template Number format. Note: there are actually 5219 video frames in this video, NOT 5184 as stated in the error. All frames are 40ms, with 96 frames per segment (ie. 3840ms per segment).

Has there been a recent update to the Dash Validator (now 1.1) which broke some math in this area which now results in reportng this error falsely!

All videos continue to play perfectly.

Please advise urgently.

waqarz commented 8 years ago

I give it a look, is the issue: https://github.com/Dash-Industry-Forum/Conformance-and-reference-source/issues/21 pointing to the same problem?

richardbushell commented 8 years ago

Yes, sorry, posted twice, wasn't sure which was correct thread, please close the wrong thread.

I look forward to your feedback on the issue reported asap.

waqarz commented 8 years ago

This seems closely related to a bug we recently closed, it was reported here: https://github.com/Dash-Industry-Forum/Conformance-and-reference-source/pull/18 and we provided a patch here: https://github.com/Dash-Industry-Forum/Conformance-and-reference-source/commit/38e41820bc04a4bf3393016b9984886d3ba35c33

The patch was regression tested against 25 vectors and showed no problems:

http://dash.edgesuite.net/dash264/TestCases/1a/qualcomm/1/MultiRate.mpd http://dash.edgesuite.net/dash264/TestCases/1b/qualcomm/2/MultiRate.mpd http://dash.edgesuite.net/dash264/TestCases/1c/qualcomm/1/MultiRate.mpd http://dash.edgesuite.net/dash264/TestCases/2a/qualcomm/1/MultiResMPEG2.mpd http://dash.edgesuite.net/dash264/TestCases/2b/qualcomm/2/MultiRes.mpd http://dash.edgesuite.net/dash264/TestCases/2c/qualcomm/1/MultiResMPEG2.mpd http://dash.edgesuite.net/dash264/TestCases/2c/qualcomm/2/MultiRes.mpd http://dash.edgesuite.net/dash264/TestCases/3a/fraunhofer/aac-lc_stereo_without_video/ElephantsDream/elephants_dream_audio_only_aaclc_stereo_sidx.mpd http://dash.edgesuite.net/dash264/TestCases/3b/fraunhofer/aac-lc_stereo_with_video/ElephantsDream/elephants_dream_480p_aaclc_stereo_sidx.mpd http://dash.edgesuite.net/dash264/TestCases/5b/nomor/6.mpd http://dash.edgesuite.net/dash264/TestCases/5b/nomor/8.mpd http://dash.edgesuite.net/dash264/TestCases/5b/nomor/9.mpd http://dash.edgesuite.net/dash264/TestCases/9a/qualcomm/1/MultiRate.mpd http://dash.edgesuite.net/dash264/TestCases/9b/qualcomm/1/MultiRate.mpd http://dash.edgesuite.net/dash264/TestCases/9c/qualcomm/1/MultiRate.mpd http://dash.edgesuite.net/dash264/TestCasesHD/1a/qualcomm/1/MultiRate.mpd http://dash.edgesuite.net/dash264/TestCasesHD/1b/qualcomm/2/MultiRate.mpd http://dash.edgesuite.net/dash264/TestCasesHD/1c/qualcomm/1/MultiRate.mpd http://dash.edgesuite.net/dash264/TestCasesHD/2b/DTV/1/live.mpd http://dash.edgesuite.net/dash264/TestCasesHD/2c/qualcomm/1/MultiResMPEG2.mpd http://dash.edgesuite.net/dash264/TestCasesMCA/fraunhofer/HE-AACv2_Multichannel/1/6chID/6chId_480p_single_adapt_heaac5_1_sidx.mpd http://dash.edgesuite.net/dash264/TestCasesMCA/fraunhofer/HE-AACv2_Multichannel/2/8chID/8ch_id_480p_single_adapt_heaac7_1_cf12_sidx.mpd http://dash.edgesuite.net/dash264/TestCasesMCA/fraunhofer/MPEGH_Stereo/1/sintel_audio_video_mpegh_stereo_sidx.mpd http://dash.edgesuite.net/dash264/TestCasesMCA/fraunhofer/MPEGH_51/1/sintel_audio_video_mpegh_5_1_brs_sidx.mpd http://dash.edgesuite.net/dash264/TestCasesHEVC/1a/1/TOS_OnDemand_HEVC_MultiRate.mpd http://dash.edgesuite.net/dash264/TestCasesHEVC/1a/2/BBB_OnDemand_HEVC_MultiRate.mpd

Some deep look will be required.

I have no clue why github is adding emphasis!

richardbushell commented 8 years ago

@waqarz Yes, that looks like it may well be related to this issue. I am sure it's now just reporting incorrectly, we've conformed our library successfully over the last 12 months, I am am entirely confident in the integrity of the 96 frame GOP segments. But as always - something fixed, something broken...

joywelt commented 8 years ago

@richardbushell We checked the issue and fixed it in PR #126

richardbushell commented 8 years ago

The fix doesn't seem to have been pushed to the Online Dash Validator yet: http://dashif.org/conformance.html (it still says v1.1 and obviously still reports the same error currently)

waqarz commented 8 years ago

Updated.

Please test again after clearing your browser cache.

richardbushell commented 8 years ago

Thanks, with the Online Validator all representations of both (video and audio) adaptation sets now pass (lovely green ticks), but we have other test files that still generate a final failure, e.g.:

https://living.video/MPD/test8.mpd

At the very last second before completing the validation we now get: Cross-representation validation success (on audio) - green tick Cross-representation validation error (on video) - red tick

There is no clickable log or any more information available as to why this might now fail this final check.

Don't know if this is related to the same error/fix. Just seen you've closed this issue, did you want me to re-post to a new issue#.

Please advise.