shaka-project / shaka-player

JavaScript player library / DASH & HLS client / MSE-EME player
Apache License 2.0
7.19k stars 1.34k forks source link

Ttml subtitle position different with shaka-player 3.2.0 and 4.1.1 #4381

Closed FakeSkdr closed 2 years ago

FakeSkdr commented 2 years ago

Have you read the FAQ and checked for duplicate open issues? Yes

What version of Shaka Player are you using? 4.1.1

Can you reproduce the issue with our latest release version? Yes

Can you reproduce the issue with the latest code from main? Yes

Are you using the demo app or your own custom app? A custom app

What browser and OS are you using? Chrome / MacOS

What are the manifest and license server URIs?

What configuration are you using? What is the output of player.getConfiguration()?

{ "drm": { "retryParameters": { "maxAttempts": 2, "baseDelay": 1000, "backoffFactor": 2, "fuzzFactor": 0.5, "timeout": 30000, "stallTimeout": 5000, "connectionTimeout": 10000 }, "servers": {}, "clearKeys": {}, "advanced": {}, "delayLicenseRequestUntilPlayed": false, "logLicenseExchange": false, "updateExpirationTime": 1, "preferredKeySystems": [] }, "manifest": { "retryParameters": { "maxAttempts": 3, "baseDelay": 1000, "backoffFactor": 2, "fuzzFactor": 0.5, "timeout": 15000, "stallTimeout": 5000, "connectionTimeout": 10000 }, "availabilityWindowOverride": null, "disableAudio": false, "disableVideo": false, "disableText": false, "disableThumbnails": false, "defaultPresentationDelay": 12, "dash": { "clockSyncUri": "", "ignoreDrmInfo": false, "disableXlinkProcessing": false, "xlinkFailGracefully": false, "ignoreMinBufferTime": false, "autoCorrectDrift": true, "initialSegmentLimit": 1000, "ignoreSuggestedPresentationDelay": false, "ignoreEmptyAdaptationSet": false, "ignoreMaxSegmentDuration": false, "keySystemsByURI": { "urn:uuid:1077efec-c0b2-4d02-ace3-3c1e52e2fb4b": "org.w3.clearkey", "urn:uuid:edef8ba9-79d6-4ace-a3c8-27dcd51d21ed": "com.widevine.alpha", "urn:uuid:9a04f079-9840-4286-ab92-e65be0885f95": "com.microsoft.playready.recommendation", "urn:uuid:79f0049a-4098-8642-ab92-e65be0885f95": "com.microsoft.playready.recommendation", "urn:uuid:f239e769-efa3-4850-9c16-a903c6932efb": "com.adobe.primetime" } }, "hls": { "ignoreTextStreamFailures": false, "ignoreImageStreamFailures": false, "useFullSegmentsForStartTime": false } }, "streaming": { "retryParameters": { "maxAttempts": 3, "baseDelay": 2000, "backoffFactor": 2, "fuzzFactor": 0.5, "timeout": 15000, "stallTimeout": 5000, "connectionTimeout": 10000 }, "rebufferingGoal": 2, "bufferingGoal": 120, "bufferBehind": 120, "ignoreTextStreamFailures": true, "alwaysStreamText": false, "startAtSegmentBoundary": false, "gapDetectionThreshold": 0.1, "smallGapLimit": 0.5, "jumpLargeGaps": false, "durationBackoff": 1, "forceTransmuxTS": false, "safeSeekOffset": 5, "stallEnabled": true, "stallThreshold": 1, "stallSkip": 0.1, "useNativeHlsOnSafari": false, "inaccurateManifestTolerance": 2, "lowLatencyMode": false, "autoLowLatencyMode": false, "forceHTTPS": false, "preferNativeHls": false, "updateIntervalSeconds": 1 }, "offline": { "usePersistentLicense": true }, "abr": { "enabled": true, "useNetworkInformation": true, "defaultBandwidthEstimate": 819200, "switchInterval": 8, "bandwidthUpgradeTarget": 0.85, "bandwidthDowngradeTarget": 0.95, "restrictions": { "minWidth": 0, "maxWidth": null, "minHeight": 0, "maxHeight": null, "minPixels": 0, "maxPixels": null, "minFrameRate": 0, "maxFrameRate": null, "minBandwidth": 0, "maxBandwidth": null } }, "preferredAudioLanguage": "qad", "preferredTextLanguage": "", "preferredVariantRole": "alternate", "preferredTextRole": "", "preferredAudioChannelCount": 2, "preferredVideoCodecs": [], "preferredAudioCodecs": [], "preferForcedSubs": false, "preferredDecodingAttributes": [], "restrictions": { "minWidth": 0, "maxWidth": null, "minHeight": 0, "maxHeight": null, "minPixels": 0, "maxPixels": null, "minFrameRate": 0, "maxFrameRate": null, "minBandwidth": 0, "maxBandwidth": null }, "playRangeStart": 0, "playRangeEnd": null }

What did you do?

I started a video associated with a ttml subtiles track using shaka-player 4.1.1

On shaka-player 3.2.0: Subtitles are bottom centered (Expected) On shaka-player 4.1.1: Subtitles are top centered (Should be bottom centered like with shaka 3.2.0)

The subtitles shouldn't change position depending on the shaka-player version

What did you expect to happen? Subtitle track is visible and bottom centered like it is on shaka-player 3.2.0

What actually happened?

The subtitles are visible, but the html box it top centered.

The TTML source:

<tt xmlns="http://www.w3.org/ns/ttml" xmlns:ttm="http://www.w3.org/ns/ttml#metadata" xmlns:ttp="http://www.w3.org/ns/ttml#parameter" xmlns:tts="http://www.w3.org/ns/ttml#styling"  xmlns:ebuttm="urn:ebu:tt:metadata" xmlns:ebutts="urn:ebu:tt:style" xml:lang="fr" xml:space="default" ttp:cellResolution="50 30" ttp:timeBase="media">
  <head>
    <metadata><ttm:title></ttm:title><ttm:desc></ttm:desc><ttm:copyright></ttm:copyright></metadata>
    <styling>
        <style xml:id="WhiteOnBlack" tts:backgroundColor="#00000080" tts:color="#ffffffff" />
        <style xml:id="defaultStyle" tts:backgroundColor="#00000000" tts:color="#ffffffff" tts:fontFamily="monospaceSansSerif" tts:fontSize="130%" tts:fontStyle="normal" tts:fontWeight="normal" tts:lineHeight="150%" tts:opacity="1" tts:textAlign="center" tts:textDecoration="none" />
        <style xml:id="doubleHeight" tts:fontSize="115%" />
        <style xml:id="textAlignCenter" tts:textAlign="center" /> 
    </styling>
    <layout>
        <region xml:id="bottomAligned" tts:displayAlign="after" tts:extent="80% 80%" tts:origin="10% 10%" tts:padding="-2%" tts:writingMode="lrtb" />
    </layout>
  </head>

  <body>
    <div style="defaultStyle" xml:id="SGN0">
      <p begin="00:00:05.560" end="00:00:07.920" region="bottomAligned" style="textAlignCenter" xml:id="sub1"><span style="WhiteOnBlack doubleHeight">Harry Potter n&apos;est pas stupide.</span></p>
    </div>
  </body>
</tt>

The html generated for shaka-player 3.2.0 (Working case):

<div class="shaka-text-container" style="text-align: center; display: flex; flex-direction: column; align-items: center; justify-content: flex-end;"><div style="white-space: pre-wrap; direction: ltr; opacity: 1; display: flex; flex-direction: column; align-items: center; justify-content: flex-end; margin: 0px; font-family: inherit; font-weight: inherit; font-style: inherit; letter-spacing: inherit; font-size: inherit; height: 80%; width: 80%; position: absolute; top: 10%; left: 10%; text-align: center; writing-mode: horizontal-tb;"><span class="shaka-text-wrapper"><span style="white-space: pre-wrap; background-color: rgba(0, 0, 0, 0.5); color: rgb(255, 255, 255); direction: ltr; opacity: 1; display: inline; font-family: inherit; font-weight: inherit; font-style: inherit; letter-spacing: inherit; font-size: inherit; text-align: start; writing-mode: horizontal-tb;">Harry Potter n'est pas stupide.</span></span></div></div>

The html generated for shaka-player 4.1.1 (Non-working case): I can see a lot of added span compared to the parsing of shaka-player 3.2.0

<div class="shaka-text-container" style="text-align: center; display: flex; flex-direction: column; align-items: center; justify-content: flex-end;"><div style="white-space: pre-wrap; direction: ltr; opacity: 1; display: flex; flex-direction: column; align-items: center; justify-content: flex-end; margin: 0px; font-family: inherit; font-weight: inherit; font-style: inherit; letter-spacing: inherit; font-size: inherit; text-align: center; writing-mode: horizontal-tb;"><span class="shaka-text-wrapper"><span style="white-space: pre-wrap; color: rgb(255, 255, 255); direction: ltr; opacity: 1; background-color: rgba(0, 0, 0, 0); display: flex; flex-direction: column; align-items: center; justify-content: flex-end; margin: 0px; font-family: inherit; font-weight: inherit; font-style: inherit; letter-spacing: inherit; font-size: inherit; text-align: center; writing-mode: horizontal-tb;"><span style="white-space: pre-wrap; direction: ltr; opacity: 1; background-color: rgba(0, 0, 0, 0); display: inline; margin: 0px; font-family: inherit; font-weight: inherit; font-style: inherit; letter-spacing: inherit; font-size: inherit; height: 80%; width: 80%; position: absolute; top: 10%; left: 10%; text-align: center; writing-mode: horizontal-tb;"><span style="white-space: pre-wrap; color: rgb(255, 255, 255); direction: ltr; opacity: 1; display: inline; font-family: inherit; font-weight: inherit; font-style: inherit; letter-spacing: inherit; font-size: inherit; text-align: center; writing-mode: horizontal-tb;"><span style="background-color: rgba(0, 0, 0, 0.5);">Harry Potter n'est pas stupide.</span></span></span></span></span></div></div>

I wonder if the change made in this PR (included in 4.x) made my ttml non-valid according to the new parser rules, but I don't understand why, and how to fix it: https://github.com/shaka-project/shaka-player/commit/bf67d87387b1dfc4d3d8e0661bfe4efb1e4083b2

Is it the expected behavior after the TTMLParser change? If it's the case, do you know what I can change on the ttml format to make it work?

Thanks for you help on this 🙏

joeyparrish commented 2 years ago

Good questions. This is what an independent TTML implementation thinks should happen:

4xgyv5bNkj5UxoG

And I agree with the sentiment.

We'll take a look. Thanks!

joeyparrish commented 2 years ago

PR #4407 fixed native browser rendering, but not the DOM-based rendering of our UI. Reopening.

joeyparrish commented 2 years ago

Here's the quick repro in JS console of the demo app:

data = shaka.util.StringUtils.toUTF8(`<tt xmlns="http://www.w3.org/ns/ttml" xmlns:ttm="http://www.w3.org/ns/ttml#metadata" xmlns:ttp="http://www.w3.org/ns/ttml#parameter" xmlns:tts="http://www.w3.org/ns/ttml#styling"  xmlns:ebuttm="urn:ebu:tt:metadata" xmlns:ebutts="urn:ebu:tt:style" xml:lang="fr" xml:space="default" ttp:cellResolution="50 30" ttp:timeBase="media">
  <head>
    <metadata><ttm:title></ttm:title><ttm:desc></ttm:desc><ttm:copyright></ttm:copyright></metadata>
    <styling>
        <style xml:id="WhiteOnBlack" tts:backgroundColor="#00000080" tts:color="#ffffffff" />
        <style xml:id="defaultStyle" tts:backgroundColor="#00000000" tts:color="#ffffffff" tts:fontFamily="monospaceSansSerif" tts:fontSize="130%" tts:fontStyle="normal" tts:fontWeight="normal" tts:lineHeight="150%" tts:opacity="1" tts:textAlign="center" tts:textDecoration="none" />
        <style xml:id="doubleHeight" tts:fontSize="115%" />
        <style xml:id="textAlignCenter" tts:textAlign="center" /> 
    </styling>
    <layout>
        <region xml:id="bottomAligned" tts:displayAlign="after" tts:extent="80% 80%" tts:origin="10% 10%" tts:padding="-2%" tts:writingMode="lrtb" />
    </layout>
  </head>

  <body>
    <div style="defaultStyle" xml:id="SGN0">
      <p begin="00:00:05.560" end="00:00:07.920" region="bottomAligned" style="textAlignCenter" xml:id="sub1"><span style="WhiteOnBlack doubleHeight">Harry Potter n&apos;est pas stupide.</span></p>
    </div>
  </body>
</tt>`);

uri = `data:application/ttml+xml;base64,${shaka.util.Uint8ArrayUtils.toBase64(data)}`;
player = video.ui.getControls().getLocalPlayer();
track = await player.addTextTrackAsync(uri, 'en', 'subtitles', "application/ttml+xml", null, null, false);

player.selectTextTrack(track);
video.currentTime = 6;
video.pause();
joeyparrish commented 2 years ago

Now fixed in both contexts.