james-cohen commented 8 months ago

Bug

I am trying to show captions on my videos using the textTracks and selectedTextTrack properties. I am using VTT as per guidance in the documentation as it is the only type supported by iOS and Android.

On Android, I have no issues and it works perfectly. On iOS, there are several serious issues if textTracks and selectedTextTrack are provided - they resolve if removed:

Videos freeze and have to be reloaded several times
When they do play, they don't have any sound
Only the first entry of the VTT file shows. When its timeframe has ended, sometimes it disappears, sometimes it stays permanently.

The VTT file is being served from a signed S3 URL. By the fact that it works on Android and occasionally shows on iOS, I am assuming that the URL itself is not the problem.

Platform

iOS

Environment info

Library version: 6.0.0-beta.3 Device: iPhone 11, iOS 17.2

Steps To Reproduce

Watch streamed MP4 video with streamed VTT subtitles

Expected behaviour

Video plays with sound
Subtitles update at the correct timestamps

Reproducible sample code

        <Video
          style={videoStyle}
          ref={ref}
          source={{ uri: src }}
          resizeMode={resizeMode}
          posterResizeMode={posterResizeMode}
          automaticallyWaitsToMinimizeStalling
          disableFocus
          useTextureView={false}
          playWhenInactive
          paused={isPaused || isSeeking}
          ignoreSilentSwitch={IgnoreSilentSwitchType.IGNORE}
          controls={!!showControls}
          repeat={!!shouldRepeat}
          muted={!!isMuted}
          poster={loadingImage}
          onReadyForDisplay={onReady}
          onLoad={handleLoad}
          onBuffer={(ev) => setBuffering(ev.isBuffering)}
          onError={(err) => handleError(err.error)}
          onEnd={handleFinish}
          progressUpdateInterval={50}
          onProgress={handleProgress}
          selectedTextTrack={
            showCaptions && transcriptUri
              ? { type: SelectedTrackType.INDEX, value: 0 }
              : { type: SelectedTrackType.DISABLED }
          }
          textTracks={
            transcriptUri
              ? [
                  {
                    title: 'Closed captions',
                    uri: transcriptUri,
                    language: 'en',
                    type: TextTracksType.VTT,
                  },
                ]
              : undefined
          }
          subtitleStyle={subtitleStyle}
          bufferConfig={{
            minBufferMs: 2500,
            maxBufferMs: 60000,
            bufferForPlaybackMs: 2500,
            bufferForPlaybackAfterRebufferMs: 2500,
            maxHeapAllocationPercent: 0.5,
          }}
        />

I have tried with 2 versions of the same VTT data, one with optional style and index properties set and one without. Both result in the same issues:

With style and order properties set:

WEBVTT

1
00:00:00.000 --> 00:00:05.760 align:middle line:50% position:50% size:80%
This is a test recording that I'm going to make at least 10 to 15 seconds long because I want it to

2
00:00:05.760 --> 00:00:10.880 align:middle line:50% position:50% size:80%
pick up on my words and do a full transcription of everything that I'm saying. I want to see how

3
00:00:10.880 --> 00:00:17.520 align:middle line:50% position:50% size:80%
well that it works and to see that the subtitles are actually prepared to show at the right time.

Without:

WEBVTT

1
00:00:00.000 --> 00:00:05.760
This is a test recording that I'm going to make at least 10 to 15 seconds long because I want it to

2
00:00:05.760 --> 00:00:10.880
pick up on my words and do a full transcription of everything that I'm saying. I want to see how

3
00:00:10.880 --> 00:00:17.520
well that it works and to see that the subtitles are actually prepared to show at the right time.

brycnguyen commented 8 months ago

@james-cohen I had similar issues on iOS. Could you try https://github.com/react-native-video/react-native-video/pull/3461 and see if that resolves ur issues?

james-cohen commented 8 months ago

Hi @brycnguyen, unfortunately not. It's now playing without freezing, but the sound is still missing and the text tracks aren't loading at all. This is the response body from onLoad:

{
  "textTracks": [],
  "canStepForward": true,
  "canPlayFastForward": true,
  "canPlaySlowReverse": true,
  "audioTracks": [],
  "currentTime": 0,
  "canPlayReverse": true,
  "duration": 17.753999710083008,
  "canPlaySlowForward": true,
  "target": 1719,
  "canStepBackward": true,
  "naturalSize": {
    "height": 1920,
    "width": 1080,
    "orientation": "landscape"
  }
}

I've confirmed in a separate console log that the transcriptionUri is definitely being supplied

castdrian commented 7 months ago

for me passing texttracks just breaks playback entirely

manyaagarwal commented 6 months ago

Hey, I am facing a similar issue where the video doesn't work at all with the following props:

 selectedTextTrack={{
          type: 'language',
          value: 'en',
        }}
        textTracks={[
          {
            title: 'English',
            language: 'en',
            type: TextTracksType.VTT,
            uri: 'https://bitdash-a.akamaihd.net/content/sintel/subtitles/subtitles_en.vtt',
          },
          {
            title: 'Deutsch',
            language: 'de',
            type: TextTracksType.VTT,
            uri: 'https://bitdash-a.akamaihd.net/content/sintel/subtitles/subtitles_en.vtt',
          },
        ]}

paulrinaldi commented 6 months ago

With a known .VTT file that works where the values on the

... textTracks={ [{"language": "en", "title": "English", "type": "text/vtt", "uri": file:///Users/paulrinaldi/Library/Developer/CoreSimulator/Devices/<ID>/data/Containers/Data/Application/<ANOTHER_ID>/Documents/captions.vtt" }] } selectedTextTrack={{"type": "title", "value": "English"}} ...

when I enable captions (i.e. set the selectedTextTrack) on iOS in 6.0.0-beta.5 the video becomes a black screen and the video and audio stop. on android in 6.0.0-beta.5 the video keeps going but no captions are shown.

KrzysztofMoch commented 6 months ago

Cloud someone test this PR #3557 ?

sntg-p commented 4 months ago

I'm also having freezes when using a VTT.

In my case, the video uses a signed S3 URL but VTT is public. It seems that if both are public URLs, the video plays properly but subtitles are currently not appearing on v6.0.0-rc.0.

Feeeliperesende commented 3 months ago

Same problem here, any solution? On Android everything works ok, but on iOS the VTT subtitles do not appear at all

KrzysztofMoch commented 3 months ago

Are you able to reproduce this in sample
If you are using hls streams it will not work

Feeeliperesende commented 3 months ago

IOS