search

Dash-Industry-Forum / dash.js

A reference client implementation for the playback of MPEG DASH via Javascript and compliant browsers.
http://reference.dashif.org/dash.js/nightly/samples/dash-if-reference-player/index.html
Other
5.1k stars 1.67k forks source link

fragmented wvtt text not recognized as fragmentedText #3685

Closed phloxic closed 3 years ago

phloxic commented 3 years ago
Environment
Steps to reproduce

Load stream with fragmented wvtt text in reference client with v4.x or latest

Observed behavior

Subtitles are not displayed. player.getCurrentTrackFor('text').isFragmented yields false

Console output

Filtering for 'text' on a minimal demo page:

[100][ScheduleController][text] Quality has changed, get init request for representationid = 1 
Debug.js:169
[160][StreamProcessor][text] OnFragmentLoadingCompleted for stream id defaultId_0 and media type text - Url: https://d12zt1n3pd4xhr.cloudfront.net/dev/lshtrailer-od/dash/lshtrailer-od-vtt_dashinit.mp4 , Range:0-672
Expected behavior

It should show the subtitles.

phloxic commented 3 years ago

FWIW, using the full profile:

dsilhavy commented 3 years ago

This should be fixed in nightly:

[623][BufferController][fragmentedText] Clearing playback buffer to overcome quota exceed situation

I could not reproduce this, when do you see this error:

Uncaught (in promise) TypeError: e.chunk is undefined

Also checking your first issue with

player.getCurrentTrackFor('text').isFragmented yields false
phloxic commented 3 years ago

I may have caused confusiong by offering a sample stream with a different profile. I'll try to clarify based on nightly (progress for 2. full profile):

  1. https://d12zt1n3pd4xhr.cloudfront.net/dev/lshtrailer-od.mpd urn:mpeg:dash:profile:isoff-on-demand:2011,http://dashif.org/guidelines/dash264 No errors, no visible subtitles. player.getCurrentTrackFor('text').isFragmented on my minimal sample page yields false (the sample now shows this on the page when text tracks are added) - the sample page still shows latest, not nightly. So for this onDemand profile the subtitles are not recognized as fragmented.

  2. https://d12zt1n3pd4xhr.cloudfront.net/dev/lshtrailer-d1.mpd urn:mpeg:dash:profile:full:2011 shows subtitles, recognizes the track as fragmented, but still gives the following error:

    [SourceBufferSink][text] SourceBuffer append failed "TypeError: VTTCue constructor: Argument 1 is not a finite floating-point value."

    The other error is gone.

dsilhavy commented 3 years ago

@phloxic Can you please check if the fix by @bbert #3689 solves your issue.

phloxic commented 3 years ago

Chrome:

[SourceBufferSink][text] SourceBuffer append failed "TypeError: Failed to construct 'VTTCue': The provided double value is non-finite."

Firefox:

[SourceBufferSink][text] SourceBuffer append failed "TypeError: VTTCue constructor: Argument 1 is not a finite floating-point value."

Safari:

[SourceBufferSink][text] SourceBuffer append failed "TypeError: The provided value is non-finite"

All browsers Mac OS, if that matters.

phloxic commented 3 years ago

@dsilhavy - https://github.com/Dash-Industry-Forum/dash.js/pull/3690/commits/27e3a78e528e07fec36d3a54d9733c693bcdcb7f makes sense and looks promising

phloxic commented 3 years ago

@dsilhavy just testing fresh nightly https://github.com/Dash-Industry-Forum/dash.js/commit/dd7eb88140d761d5653223e87d6f945872546bbb with full profile: Almost there, I think. The error is now (in all browsers I quickly checked so far):

[TextTracks] impossible to display subtitles.

I presume that's when an isNaN is hit.

dsilhavy commented 3 years ago

Yes I think this is fine as the error is catched now and we should inform the app about an invalid cue.

phloxic commented 3 years ago

So the conformance tool should bark at this stream then? It doesn't. Although the test is quicker, and less extensive by the looks of it. Not sure it actually segment validation in spite of the input summary reporting it:

MPD :
https://d12zt1n3pd4xhr.cloudfront.net/dev/lshtrailer-d1.mpd

Additional Tests :
Segment Validation
DASH-IF

Also weird: dash.js now handles the isoff-on-demand profile fine and without error, in spite of the conformance tool complaining about the text track.

My guess is that in both profiles the offset of the first cue (the error only occurs at the beginning) is in certain circumstances considered as not properly signaled by the mp4, which is, for isoff-on-demand profile, noticed by the conformance tool and either ignored or worked around by the player. Whereas for the full profile the conformance tool either ignores the problem or is satisfied, but the player isn't.

From the user point of view, there is no visible problem, no cues are missing, arguably this could be called a successful workaround (what?) instead of an 'internal' error ;-)

dsilhavy commented 3 years ago

It looks like the first caption is defined in multiple ranges:

phloxic commented 3 years ago

Looks like the 3rd caption contains the first 3 captions. Thanks for taking the time to diagnose this. I'll probably take it up with the gpac folks. If I find time that is.

Any ideas why the conformance tool doesn't do the segment validation for the full profile?

dsilhavy commented 3 years ago

Unfortunately I dont have insights in the conformance tool. I know that they are actively working on it though so it might be good input for them if something is wrong with the full profile. I am closing this issue for now as everything looks fine in dash.js. Thanks for reporting.

dsilhavy commented 3 years ago

Additional question: Will those two subtitle MPDs stay online? Then we can consider adding them to the list of reference vectors as we are missing the combination of SegmentBase and VTT

phloxic commented 3 years ago

@dsilhavy - ok, I'll leave them there, and try to remember if I change anything.

Don't know how useful the full profile is, as it's broken. Btw, I was also too hasty in claiming that works from the viewer's point of view; watching more closely the first 3 cues display sequentially, but too late and in too short sucession. FWI, I tried the simple and main profiles as well, and they exhibit the same problem as the full profile.