Support default audio and subtitle tracks

kvenn commented 8 months ago

Fixes: https://github.com/itsSagarBro/modern_player/issues/3

Summary

Adds support for automatically selecting a subtitle or audio track based on provided criteria.

Implementation

For tracks embedded in the file (automatically selected ones), we need to wait for the player to load the file first
We have the _getTracks function which is only invoked when first loaded or a refresh is required (via changeVidoeQuility)
- Note: this is only technically true given the current implementation. On an adaptive stream, you could change quality but not be required to reload the tracks.
- It might be worth preventing the track from changing if one is already selected (and the subtitles don't change) in the future.
Automatically select the default track based on the provided criteria, or no-op

This PR does not add quality selection. In an adaptive stream, it'd look similar to text and audio, but I believe this is using multiple provided URLs.

Example

This example shows us automatically selecting a default language of "English".

ModernPlayer.createPlayer(
  controlsOptions: ModernPlayerControlsOptions(showBackbutton: false),
  defaultSelectionOptions: ModernPlayerDefaultSelectionOptions(
    defaultSubtitleSelectors: [
      DefaultSelectorCustom((key, value) =>
          value.toLowerCase().contains('english') ||
          value.toLowerCase().contains('en')),
      DefaultSelectorOff(),
    ],
    defaultAudioSelectors: [
      DefaultSelectorCustom((key, value) =>
          value.toLowerCase().contains('english') ||
          value.toLowerCase().contains('en'))
    ],
  ),
  video: ModernPlayerVideo.single(ModernPlayerVideoData.network(
    label: "Default",
    url: "myNetworkUrl",
  )));

Questions

Instead of having its own "options" class, it could have been bundled with "controls".

Future Steps

[ ] Offer an enum (or ISO 639 support) that simplifies language selection (so each person doesn't need to do the "contains" check themselves).
[x] Provide an ordered lists of "default" languages, so for example, if English isn't available it falls back to Spanish.
[ ] Add quality selection

itsSagarBro commented 8 months ago

@kvenn Thank you for the PR, I'll merge it once the testing is complete.

kvenn commented 8 months ago

This change might be more controversial because it requires adding an external dependency, but to add support for language select. With the sealed class, this should be easy to add

Dependency

dependencies:
  sealed_languages: 1.1.0

New sealed class option

class DefaultSelectorLanguage extends DefaultSelector {
  /// The ISO 639-1 (2 character) language code
  final String language;

  DefaultSelectorLanguage(this.language);
}

Usage

  final defaultEnglishOptions = ModernPlayerDefaultSelectionOptions(
    defaultSubtitleSelectors: [
      DefaultSelectorLanguage('en'),
    ],
  );

New helper

import 'package:sealed_languages/sealed_languages.dart';
import 'dart:core';

class SubtitleHelper {
  int findTrack(String isoLang, Map<int, String> map) {
    final language = NaturalLanguage.fromCodeShort(isoLang.toLowerCase());
    String? englishName = language?.name?.toLowerCase();
    List<String> nativeNames = language?.namesNative?.map((e) => e.toLowerCase()).toList() ?? [];

    int? matchIndex;
    int shortestNameLength = 99999;  // Initialize with a large number for comparison.

    // First, try to match by English name and select the one with the shortest name if multiple matches are found.
   // Shortest name to prevent selecting tracks like "English Force" or "English - Directors Commentary"
    if (englishName != null) {
      for (var entry in map.entries) {
        if (entry.value.toLowerCase().contains(englishName) && entry.value.length < shortestNameLength) {
          matchIndex = entry.key;
          shortestNameLength = entry.value.length;  // Update with the new shortest length.
        }
      }
    }

    // If no match, try to match by native names
    if (matchIndex == null) {
      for (var nativeName in nativeNames) {
        for (var entry in map.entries) {
          if (entry.value.toLowerCase().contains(nativeName)) {
            matchIndex = entry.key;
            break;
          }
        }
        if (matchIndex != null) break;
      }
    }

    // If still no match, try to match by the two-letter ISO code using regex
    if (matchIndex == null) {
      final isoCodePattern = RegExp(r'\b' + RegExp.escape(isoLang) + r'\b', caseSensitive: false);
      for (var entry in map.entries) {
        if (isoCodePattern.hasMatch(entry.value)) {
          matchIndex = entry.key;
          break;
        }
      }
    }

    return matchIndex ?? -1;
  }
}

kvenn commented 6 months ago

@itsSagarBro just curious if you have plans to merge this? I was thinking about adding some more features to this lib but wonder if I should PR them here or just start a fork.

itsSagarBro commented 6 months ago

@kvenn Thanks for the PR, and really sorry for delay, I was busy with my professional works. Now I am reviewing this PR and update you within some hour.

itsSagarBro commented 6 months ago

@kvenn What is the point of using this method of default selection for audio and subtitle, If we have already a method, which is more user-friendly than this one?

I have already given an option named "isSelected" when adding an audio or subtitle.

kvenn commented 6 months ago

That's a great question!

Certain video formats come bundled with multiple Audio or Subtitle tracks encoded right into them (like MKV, MP4, MOV, etc). In this case, I'm not actually defining the tracks myself, and ideally don't ever have to. The MKV file contains the mp3 files and the srt.

The reason I turned to modern_player was because it is backed by VLC, which is one of the few players to support MKV.

The example I included in the PR description is actually real code I'm using in my project (but with actual network URLs). I speak english so by default I want the language to default to english, and I want the subtitles to default to "off". But I'm going to give users the ability to specify this in my app's settings. I have no control over the order the MKV file will specify the audio or subtitle tracks, so added this feature to deterministically apply a selection in that case.

Since I imagine 90% of tracks to be language based, it might make it more approachable for the library to offer a handful of ISO 639 default selectors (but I included that as possible future work).

itsSagarBro commented 6 months ago

That's a great question!

Certain video formats come bundled with multiple Audio or Subtitle tracks encoded right into them (like MKV, MP4, MOV, etc). In this case, I'm not actually defining the tracks myself, and ideally don't ever have to. The MKV file contains the mp3 files and the srt.

The reason I turned to modern_player was because it is backed by VLC, which is one of the few players to support MKV.

The example I included in the PR description is actually real code I'm using in my project (but with actual network URLs). I speak english so by default I want the language to default to english, and I want the subtitles to default to "off". But I'm going to give users the ability to specify this in my app's settings. I have no control over the order the MKV file will specify the audio or subtitle tracks, so added this feature to deterministically apply a selection in that case.

Since I imagine 90% of tracks to be language based, it might make it more approachable for the library to offer a handful of ISO 639 default selectors (but I included that as possible future work).

@kvenn Sounds good.

But I think we can also use a simple list of string instead of list of classes or callbacks, because we don't have any major need of index based checking, and a simple list of string which compare only label is more user-friendly.

SizedBox(
  height: 250,
  child: ModernPlayer.createPlayer(
    defaultSelectionOptions: ModernPlayerDefaultSelectionOptions(
        defaultAudioSelectors: ["English", "Hindi"]),
    video: ModernPlayerVideo.single(ModernPlayerVideoData.network(
        label: 'Default',
        url:
            'https://pub-6e381cd86ecb493082a4ec40d9538030.r2.dev/The%20Boy%20the%20Mole%20the%20Fox%20and%20the%20Horse%20(2022)%20Dual%20Audio%20%7BHindi-English%7D%20480p%20WEB-DL%20ESubs%20%5BBollyFlix%5D.mkv')),
  ),
)

class ModernPlayerDefaultSelectionOptions {
  List<String>? defaultSubtitleSelectors;
  List<String>? defaultAudioSelectors;
  List<String>? defaultQualitySelectors;

  ModernPlayerDefaultSelectionOptions(
      {this.defaultSubtitleSelectors,
      this.defaultAudioSelectors,
      this.defaultQualitySelectors});
}

/// Helper function to set default track for subtitle, audio, etc
Future<void> _setDefaultTrack(
    {required List<String>? selectors,
    required Map<int, String>? trackEntries,
    required Function(int) setTrackFunction}) async {
  if (selectors == null || trackEntries == null || trackEntries.isEmpty) {
    return;
  }

  for (final selector in selectors) {
    int? defaultIndex;
    for (final entry in trackEntries.entries) {
      if (entry.value.toLowerCase().contains(selector.toLowerCase())) {
        defaultIndex = entry.key;
        break;
      }
    }

    if (defaultIndex != null) {
      setTrackFunction(defaultIndex);
      return;
      // Else, if no track is found, loop to the next selector
    }
  }
}

Just like this.

kvenn commented 6 months ago

This is your library, so it's up to you (of course).

I wrote this as an option that's the least opinionated and exposes the max ability to consumers. By allowing someone to pass in the function they can provide any logic they want. Perhaps I want to set a default based on the index of the subtitle / audio track. Or I have a specific substring within the audio track I want to compare against (prefixed by en_, for example).

The current sealed class approach should easily support unlimited more customization (given that the building blocks are abstract). To make it easier for anyone to use the "String contains", you could have something like the following. Note that it just extends DefaultSelectorCustom. So we could add as many specific types as we want (to make ti easier for consumers), while still offering a generic approach for more complicated use cases.

/// A more opinionated DefaultSelectorCustom that gets initialized with a list
/// of strings and checks if the lowercase label contains any of them
class DefaultSelectorContains extends DefaultSelectorCustom {
  final List<String> contains;

  DefaultSelectorString(this.contains)
      : super((index, label) => contains.any(
            (element) => label.toLowerCase().contains(element.toLowerCase())));
}

Use via:

defaultSubtitleSelectors: [
      DefaultSelectorString(["English", "Hindi"]),
      DefaultSelectorOff(),
    ],

Or more directly solving this problem, I was recommending we eventually add ISO 639 so all the languages are enumerated.

defaultSubtitleSelectors: [
      DefaultSelectorLanguage([LangEng(), LangSpa()]),
      DefaultSelectorOff(),
    ],

itsSagarBro commented 6 months ago

@kvenn Thank for the PR and the efforts you have given for improving this package, I appreciate your work. All the things are working perfectly which you have implemented.

Can you please add one more commit with implementing the DefaultSelectorString and change the name of it to DefaultSelectorLabel.

defaultSubtitleSelectors: [
      DefaultSelectorLabel("English"),
      DefaultSelectorOff(),
    ],

So we can also use it like this. After this updated commit, I'll approve this and merge it. Once again, thanks and sorry for my bad english.

kvenn commented 6 months ago

Cool! That sounds good to me.

I'll let you know when the change is made. Thanks for chatting through it with me.

kvenn commented 6 months ago

Updated!

itsSagarBro commented 6 months ago

@kvenn Thank you for the support and helping me to improve this package.

itsSagarBro / modern_player