[Proposal] Add Subtitle Support to Media Element

Feature name

Media Element Subtitles

Link to discussion

https://github.com/CommunityToolkit/Maui/discussions/1867

Progress tracker

[X] Android Implementation
[X] iOS Implementation
[X] MacCatalyst Implementation
[X] Windows Implementation
[ ] Tizen Implementation
[ ] Unit Tests
[x] Samples
[ ] Documentation

Summary

Add support for Subtitles to Media Element. Initial support for Windows, Mac, iOS, and Android. Subtitle format allowed at launch include srt and sub.

Windows

windows-subtitle-normal

Windows-subtitle-fullscreen

Android

Android-normal-subtitles

Android-subtitles-fullscreen

IOS

https://github.com/CommunityToolkit/Maui/assets/4167863/ab3c3f09-ddb4-4ba4-a8db-f84efba15ab9

Motivation

I would like to help support people who have difficulty with hearing or need assistance in loud noisy environments where it is hard to hear the dialogue. Or people that want translations of foreign languages spoken that they do not know. Other people just like subtitles. This will add robust support that can be expanded later.

Detailed Design

API:

    /// <summary>
    /// Backing store for the <see cref="SubtitleFont"/> property.
    /// </summary>
    public static readonly BindableProperty SubtitleFontProperty = BindableProperty.Create(nameof(SubtitleFont), typeof(string), typeof(MediaElement), string.Empty);

    /// <summary>
    /// Backing store for the <see cref="SubtitleFontSize"/> property.
    /// </summary>
    public static readonly BindableProperty SubtitleFontSizeProperty = BindableProperty.Create(nameof(SubtitleFontSize), typeof(double), typeof(MediaElement), 16.0);

    /// <summary>
    /// Backing store for the <see cref="SubtitleUrl"/> property.
    /// </summary>
    public static readonly BindableProperty SubtitleProperty = BindableProperty.Create(nameof(SubtitleUrl), typeof(string), typeof(MediaElement), string.Empty);

Usage Syntax

XAML Usage:


 <toolkit:MediaElement
     x:Name="MediaElement"
     ShouldAutoPlay="True"
     SubtitleFont="monospace"
     SubtitleFontSize="20"
     SubtitleUrl="https://raw.githubusercontent.com/ne0rrmatrix/SampleVideo/main/SRT/WindowsVideo.srt"
     Source="https://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4"/>

C# Code behind example

MediaElement.SubtitleFont = "monospace";
MediaElement.SubtitleFontSize = 16;
MediaElement.SubtitleUrl = "https://raw.githubusercontent.com/ne0rrmatrix/SampleVideo/main/SRT/WindowsVideo.srt";
MediaElement.Source = MediaSource.FromResource("WindowsVideo.mp4");

Drawbacks

Currently it uses a custom class to read the subtitle files and convert them to strings that can be used at specific times in a text box appropriate for each OS. This is a new class that only supports plain text at the moment. Non standard files and RTF formatted files are not supported.

Alternatives

The custom class for reading subtitle files was written to allow iOS and Mac Catalyst support as the current methods are to use m3u8 files and do not support directly linking a subtitle file. ATM we have support for m3u8 files but we have no control over how or what is displayed at all. Android and Windows both have robust libraries that support subtitles. They do not support any common format of subtitles.

ATM if you want uniform support for multiple different formats on each and every device my custom class is the way to go.

Unresolved Questions

How are we going to use/add fonts? Are we going to go the normal route and just add fonts to the project and import them like any other maui project? Are we going to just use built in supported OS fonts for each device?

CommunityToolkit / Maui