subtitles are not showing in videojs (#react.js #typescript #AzureMediaServices #DRM)

[Parvat12]

Description

subtitles are not showing in videojs

I am using video.js in our react.js and typescript project. for streaming we are using azure media services and DRM for encryption. video is playing perfectly and also multiple audio is palying fine but the issue we are facing is subtitle is not showing. here is the code we are using but in this.player.textTracks() function is giving no TextTracks. its working fine with shaka player and subtitles are showing also its working fine in react native but the player is diffrent and that is not supported for web.

import videojs from 'video.js';
import '@videojs/http-streaming';
import "videojs-contrib-eme";
import HlsParser from 'hls-parser';

componentDidMount() {
      this.player = this.initializePlayer();

        // set eme plugin
        this.player.eme();

        // add video resources streaming url etc
        this.setVideoReaource();

        this.fetchSubtitles();
    }

     initializePlayer = () => {
        return videojs("video", {
            enableSourceset: true,
            preload: "auto",
            fluid: true,
            aspectRatio: '16:9',
            controls: true,
            autoplay: true,
            controlBar: {
                pictureInPictureToggle: false,
                remainingTimeDisplay: false,
                fullscreenToggle: true,
                volumePanel: {
                    inline: false
                },
            },
            html5: {
                nativeTextTracks: true
            }
        });
    }

    setVideoReaource = () => {
        this.player.src({
            src: "https://vodemedia-usea.streaming.media.azure.net/4ac3e64a-aa82-460d-a7f5-06632522aae2/The Bad Guys_2022 New Eng and Ar.ism/manifest(format=m3u8-cmaf,encryption=cenc)",
            type: 'application/x-mpegURL',
            keySystems: {
                'com.widevine.alpha': this.widevineLicenseURI,
                'com.microsoft.playready': true
            },
        });
    }

    fetchSubtitles() {
        const subtitles = this.player.textTracks();
        console.log('subtitles');
        console.log(subtitles);
        for (const element of subtitles) {
          const subtitle = element;
          if (subtitle.src) {
            this.player.addRemoteTextTrack({
              kind: 'subtitles',
              src: subtitle.src,
              srclang: subtitle.language,
              label: subtitle.label,
              default: true
            });
          }
        }
      }

      render() {
        return(
          <div>
             <video
               height="480"
               id="video"
               className={`video-js vjs-default-skin vjs-fill vjs-custom-theme`}
               controls
               data-setup="{}">
             </video>     
          </div>
      }

Note:- also we try to fetch subtitle from manifest url using this code but we are not getting any playlists in manifestObject.

fetch("https://vodemedia-usea.streaming.media.azure.net/4ac3e64a-aa82-460d-a7f5-06632522aae2/The Bad Guys_2022 New Eng and Ar.ism/manifest(format=m3u8-cmaf,encryption=cenc)")
            .then(response => response.text())
            .then(manifest => {
                console.log(manifest);
                const manifestObject = HlsParser.parse(manifest);
                console.log('manifestObject');
                console.log(manifestObject);

                   const subtitlePlaylist = manifestObject.playlists.find(playlist => playlist.attributes && playlist.attributes.SUBTITLES);
                   const subtitleTrack = subtitlePlaylist && subtitlePlaylist.mediaGroups.SUBTITLES.en.find(track => track.default);

                    console.log('subtitlePlaylist');
                    console.log(subtitlePlaylist);
                    this.player.addRemoteTextTrack({
                        kind: 'subtitles',
                        src: subtitleTrack.uri,
                        srclang: subtitleTrack.language,
                        label: 'English'
                      });
            })
        .catch(error => {
            console.error(error);
        });

please help me if any one have solution on this. and if want to connect one to one please let me know we can connect.

Reduced test case

https://localvodsaudi-118668-react-native.b118668.dev.eastus.az.svc.builder.cafe/MovieDetails/32

Steps to reproduce

1.use the code 2.initialize the player class or play the video 3.subtitle options will show and video will play but subtite will not show

Errors

No response

What version of Video.js are you using?

7.20.3

Video.js plugins used.

@videojs/http-streaming, videojs-contrib-eme

What browser(s) including version(s) does this occur with?

chrome Version 107.0.5304.121

What OS(es) and version(s) does this occur with?

ubuntu

[welcome[bot]]

👋 Thanks for opening your first issue here! 👋

If you're reporting a 🐞 bug, please make sure you include steps to reproduce it. We get a lot of issues on this repo, so please be patient and we will get back to you as soon as we can. To help make it easier for us to investigate your issue, please follow the contributing guidelines.

[pritesh511]

Yes, @Parvat12 I have same issue any one can help us please.

[amtins]

@Parvat12 videojs doesn't handle TTML subtitles. Maybe you can try with https://github.com/videojs/videojs-contrib-dash with https://github.com/videojs/videojs-contrib-dash#using-ttml-captions

#EXT-X-STREAM-INF:BANDWIDTH=460340,RESOLUTION=320x180,CODECS="avc1.64000d,mp4a.40.2,stpp.ttml.im1t",AUDIO="audio",SUBTITLES="subs"

Quick question, does HLS work with widevine? I haven't had time to test it properly, but I'm very interested in your feedback.

[Parvat12]

thank you @amtins , yes HLS is working fine with widevine, only issue i am facing is my subtitle vtt files are not playing with menifest url. also i checked with dash formate as well but i am not aware about TTML.

[amtins]

@Parvat12 there is the documentation about Media Services API v3 Captions vs subtitles, I guess there is a button somewhere where you can enable ttml or webvtt subtitles.

You can also have a look at the section Simple code comparison of TTML and WebVTT at the very bottom of the page.

[Parvat12]

Thanks @amtins but we are using WebVTT only and I didn't found any button for enable. same manifest url is working for shaka player as well as native player but with video.js i am not sure what is the issue also there is no error showing in the console.

[gkatsev]

The addRemoteTextTrack API requires a single VTT file and not an m3u8. Ideally, your m3u8 contains segmented vtt for the captions, since we don't support vtt embedded in mp4 containers in addition to not support IMSC/TTML.

[Parvat12]

Thanks @amtins yeah I tried that as well but the vtt url is private blob at azure side and we can't make it public coz we don't want it to be downloadable for all. here is the vtt link: https://vodmediast.blob.core.windows.net/asset-2a17426f-f2fc-4689-af70-406de9ad9913/47de0db5-8c20-4a6d-aa32-8378f6716911.vtt

[amtins]

@Parvat12, my apologize, I suggested you use videojs-contrib-dash which obviously won't work with HLS content. 😅

I requested a fragment from your playlist and got a TTML back:

//list of all subtitle fragments
fetch('https://vodemedia-usea.streaming.media.azure.net/4ac3e64a-aa82-460d-a7f5-06632522aae2/The%20Bad%20Guys_2022%20New%20Eng%20and%20Ar.ism/QualityLevels(3400)/Manifest(subt_en,format=m3u8-cmaf,encryption=cenc)').then(r => r.text()).then(console.log) 

//a random subtitle fragment
fetch('https://vodemedia-usea.streaming.media.azure.net/4ac3e64a-aa82-460d-a7f5-06632522aae2/The%20Bad%20Guys_2022%20New%20Eng%20and%20Ar.ism/QualityLevels(3400)/Fragments(subt_en=420000000,format=mpd-time-cmaf,encryption=cenc)').then(r => r.text()).then(console.log) 

The subtitle fragment response:

<?xml version="1.0" encoding="utf-8"?>
<tt xml:lang="" xmlns="http://www.w3.org/ns/ttml" xmlns:tts="http://www.w3.org/ns/ttml#styling"
  xmlns:ttm="http://www.w3.org/ns/ttml#metadata">
  <head></head>
  <body region="CaptionArea">
    <div>
      <p begin="00:00:42.000" end="00:00:44.000">You know he won't give it to you.</p>
      <p begin="00:00:44.000" end="00:00:44.120"></p>
      <p begin="00:00:44.120" end="00:00:46.000">No, I think deep down that<br />The snake is a kind
        and generous soul.</p>
      <p begin="00:00:46.000" end="00:00:48.000">No, I think deep down that<br />The snake is a kind
        and generous soul.</p>
    </div>
  </body>
</tt>

The VTT file is the asset you provided to Azure to ingest, but it is not necessarily the same type of file that you will get after the packaging is complete.

[Parvat12]

thankyou @amtins yes I tried dash its working as you showed but again the question is how I can show these captions in my video for video.js player. currently the video is playing but subtitle is not showing.

[amtins]

@Parvat12 in my opinion, this is something that should be adjusted in the Azure admin interface. If you can't or don't know how to make the change, you can contact the Azure support team who can help you.

The section Text tracks mentions tell the player about the text tracks with the portal which unfortunately leads to a 404.

[Parvat12]

@amtins thanks for your responce but I think the problem is we are using protected vtt files thats why its not playing and if we make azure blob container (where azure storing the vtt file) public then its playing with hls i tried but we can't make it public coz we want that it should not accessible for all by using the vtt_url that can easily find out in network tab. so I think this problem is from video.js not from azure also the same protected vtt file is working fine with other players without doing any extra code.

[Parvat12]

@gkatsev could you please help us on this.