adgang commented 1 year ago

Bug

In iOS, when a video with multiple audio tracks is played and if both audio tracks are set as disposition default, then only one audio track is being considered by react-native-video for playing.

Platform

Which player are you experiencing the problem on:

iOS

In safari, the video plays fine with both audio tracks shown as options. So the issue is likely to be in this codebase.

Environment info

React native info output: FWIW we noticed the issue on older react-native(0.68) also

System:
    OS: macOS 13.0.1
    CPU: (8) arm64 Apple M1
    Memory: 70.06 MB / 16.00 GB
    Shell: 5.8.1 - /bin/zsh
  Binaries:
    Node: 16.14.2 - /private/var/folders/_j/19d37zp90s57mwb7m72352l40000gn/T/xfs-ea49d6c4/node
    Yarn: 3.2.0 - /private/var/folders/_j/19d37zp90s57mwb7m72352l40000gn/T/xfs-ea49d6c4/yarn
    npm: 8.5.0 - ~/Library/Caches/fnm_multishells/2244_1668186879933/bin/npm
    Watchman: Not Found
  Managers:
    CocoaPods: 1.11.3 - /var/folders/_j/19d37zp90s57mwb7m72352l40000gn/T/frum_2257_1668186879970/bin/pod
  SDKs:
    iOS SDK:
      Platforms: DriverKit 22.1, iOS 16.1, macOS 13.0, tvOS 16.1, watchOS 9.1
    Android SDK: Not Found
  IDEs:
    Android Studio: 2021.3 AI-213.7172.25.2113.9123335
    Xcode: 14.1/14B47b - /usr/bin/xcodebuild
  Languages:
    Java: 11.0.14.1 - /usr/bin/javac
  npmPackages:
    @react-native-community/cli: Not Found
    react: 18.1.0 => 18.1.0 
    react-native: 0.70.5 => 0.70.5 
    react-native-macos: Not Found
  npmGlobalPackages:
    *react-native*: Not Found

Library version: 5.2.1

Steps To Reproduce

Create an mp4 video with two audio tracks, both set default disposition.
Play it in react-native-video in ios. Only one track will be visible/playable in the app. ...

Expected behaviour

Player should show both audio tracks.

Reproducible sample code

  const handleOnLoad = (data: any) => {
    console.log({ data });
    console.log(data.audioTracks[0]);
    console.log(data.audioTracks[1]);
  };

<Video
  onLoad={handleOnLoad}
  source={{ uri: url }}
  style={{ width: '100%', height: '100%' }}
/>

The log showed only one audio track.

 LOG  {"data": {"audioTracks": [[Object]], "canPlayFastForward": true, "canPlayReverse": true, "canPlaySlowForward": true, "canPlaySlowReverse": true, "canStepBackward": true, "canStepForward": true, "currentTime": 0, "duration": 3.878000020980835, "naturalSize": {"height": 1080, "orientation": "landscape", "width": 1920}, "target": 3889, "textTracks": []}}
 LOG  {"index": 0, "language": "en", "title": "A"}
 LOG  undefined

Video sample

https://d3bm2z03a5d1kp.cloudfront.net/narrations/0d4197b0-7fc3-45b2-858b-05fd8977eb38/test.mp4

ffprobe of the sample:

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'test.mp4':
  Metadata:
    major_brand     : M4V 
    minor_version   : 1
    compatible_brands: isomavc1mp42
    creation_time   : 2022-11-22T17:32:24.000000Z
  Duration: 00:00:21.40, start: 0.000000, bitrate: 3376 kb/s
  Stream #0:0[0x1](und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], 3174 kb/s, 25 fps, 25 tbr, 25k tbn (default)
    Metadata:
      creation_time   : 2022-11-22T17:32:24.000000Z
      handler_name    : ETI ISO Video Media Handler
      vendor_id       : [0][0][0][0]
      encoder         : Elemental H.264
  Stream #0:1[0x2](und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 96 kb/s (default)
    Metadata:
      creation_time   : 2022-11-22T17:32:24.000000Z
      handler_name    : ETI ISO Audio Media Handler
      vendor_id       : [0][0][0][0]
  Stream #0:2[0x3](und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 96 kb/s (default)
    Metadata:
      creation_time   : 2022-11-22T17:32:24.000000Z
      handler_name    : ETI ISO Audio Media Handler
      vendor_id       : [0][0][0][0]

Notice that both the audio tracks have default set at the end.

After I did

ffmpeg -i test.mp4 -map 0:v:0 -map 0:a:0 -map 0:a:1 -c copy -disposition:a:0 default \
       -disposition:a:1 0 reordered.mp4

The reordered.mp4 plays fine in iOS.

PS: This issue is not happening in Android.

freeboub commented 11 months ago

tested on V6.0.0-akpha.8, issue still reproduced. can see audio track in VLC, no audio track in RNV sample ios

programmer1zero1 commented 6 days ago

@freeboub is there any solution cause i am having the same issue for same video url.

freeboub commented 6 days ago

@programmer1zero1 I checked again, I am not able to read the tracks from AVPlayer API... I don't have any easy solution for now. Maybe it is simply an AVPlayer issue :/

both : return try await asset.loadMediaSelectionGroup(for: mediaCharacteristic) and return asset?.mediaSelectionGroup(forMediaCharacteristic: mediaCharacteristic)

returns empty audio track list ... I tried:

   let group = await RCTVideoAssetsUtils.getMediaSelectionGroup(asset: asset, for: .audible)
    if #available(iOS 17.0, *) {
        let group2 = await RCTVideoAssetsUtils.getMediaSelectionGroup(asset: asset, for: .carriesVideoStereoMetadata)
    } else {
        // Fallback on earlier versions
    }
    let group3 = await RCTVideoAssetsUtils.getMediaSelectionGroup(asset: asset, for: .describesMusicAndSoundForAccessibility)
    let group4 = await RCTVideoAssetsUtils.getMediaSelectionGroup(asset: asset, for: .dubbedTranslation)
    if #available(iOS 17.0, *) {
        let group5 = await RCTVideoAssetsUtils.getMediaSelectionGroup(asset: asset, for: .enhancesSpeechIntelligibility)
    } else {
        // Fallback on earlier versions
    }
    let group6 = await RCTVideoAssetsUtils.getMediaSelectionGroup(asset: asset, for: .languageTranslation)
    let group7 = await RCTVideoAssetsUtils.getMediaSelectionGroup(asset: asset, for: .voiceOverTranslation)
    let group8 = await RCTVideoAssetsUtils.getMediaSelectionGroup(asset: asset, for: .transcribesSpokenDialogForAccessibility)
    let group9 = await RCTVideoAssetsUtils.getMediaSelectionGroup(asset: asset, for: .visual)
    let group10 = await RCTVideoAssetsUtils.getMediaSelectionGroup(asset: asset, for: .usesWideGamutColorSpace)
    let group11 = await RCTVideoAssetsUtils.getMediaSelectionGroup(asset: asset, for: .isOriginalContent)
    let group12 = await RCTVideoAssetsUtils.getMediaSelectionGroup(asset: asset, for: .isAuxiliaryContent)
    let group13 = await RCTVideoAssetsUtils.getMediaSelectionGroup(asset: asset, for: .frameBased)

freeboub commented 6 days ago

I may have found an issue in the code, let me triple check (rewrite audio track management in fact...)

freeboub commented 5 days ago

I can see the 2 audio tracks with : let audioTr = asset.tracks(withMediaType: .audio) But I am not able to restore ''full feature", mainly identify which track is selected ...

Ramzan034 commented 5 days ago

@freeboub Thank you! While I'm not entirely sure what the main issue was, it seems to be related to formatting. I've successfully created a video with multiple audio tracks, and it's working well on both Android and iOS.

programmer1zero1 commented 5 days ago

@freeboub could you please implementing a track selection option in the full-screen mode of the React Native video player? I currently have this feature on iOS, but it's not available on Android. Additionally, I'm also looking for a way to add speed control for the video in full-screen mode, as this feature is present on iOS but not on Android. Your help would be greatly appreciated!