TTML subtitles - Opacity not compliant with TTML specification

Xlcghs commented 8 months ago

In the TTML specification (https://www.w3.org/TR/ttml2/#style-value-border-color), section 10.3.1 , it is stated that "An expression is used to express an opacity value, where 0 means fully transparent and 1 means fully opaque." However, Media 3 uses values between 0 and 255 for alpha values. How can I make Media3 support alpha values in the form of a real number?

Source code Media 3 v1.2 :

/**

Parses a TTML color expression.
@param colorExpression The color expression.
@return The parsed ARGB color. */ @ColorInt public static int parseTtmlColor(String colorExpression) { return parseColorInternal(colorExpression, false); }

icbaker commented 8 months ago

You've linked to https://www.w3.org/TR/ttml2/#style-value-border-color which resolves to 10.3.5, but you reference section 10.3.1 in the text.

10.3.1 defines how <alpha> is used in the spec, which seems to only be referenced from tts:opacity. ExoPlayer does not currently support this tts:opacity attribute.

The media3 source code you've linked is intended to parse a TTML <color> string as defined in 10.3.9 (i.e. where the alpha component is between 0 and 255):

A \<color> expression is used to specify a named color, exact RGB color triple, or exact RGBA color tuple, where the alpha component, if expressed, is maximum (255) at 100% opacity and minimum (0) at 0% opacity, and where the applicable color space is defined by [SRGB].

I believe that all call-sites of parseTtmlColor are passing a <color> string - if you don't believe that's the case, please link to the specific call-site that you believe to be causing a problem. Please also provide a TTML file that you believe is not parsed correctly by ExoPlayer.

Xlcghs commented 8 months ago

Thanks for your feedback. Please find below an example of TTML that has caused issues with media3 :

<tt
    xmlns="http://www.w3.org/ns/ttml"
    xmlns:tts="http://www.w3.org/ns/ttml#styling"
    xmlns:ttm="http://www.w3.org/ns/ttml#metadata"
    xmlns:smpte="http://www.smpte-ra.org/schemas/2052-1/2010/smpte-tt"
    xmlns:ttp="http://www.w3.org/ns/ttml#parameter" xml:lang="fra">
    <head>
        <ttp:profile use="http://www.w3.org/ns/ttml/profile/sdp-us"/>
        <styling>
            <style xml:id="s000" tts:fontSize="67%" tts:fontFamily="proportionalSansSerif" tts:textAlign="center" tts:color="rgba(255,0,254,1.00)" tts:backgroundColor="rgba(0,0,0,0.50)"/>
            <style xml:id="s001" tts:fontSize="67%" tts:fontFamily="proportionalSansSerif" tts:textAlign="center" tts:color="rgba(255,255,255,1.00)" tts:backgroundColor="rgba(0,0,0,0.50)"/>
        </styling>
        <layout>
            <region xml:id="0_caption_7236917318434_0" tts:origin="31.707% 87.500%" tts:extent="36.585% 8.333%"/>
            <region xml:id="0_caption_7236917473234_0" tts:origin="41.463% 87.500%" tts:extent="17.073% 8.333%"/>
        </layout>
    </head>
    <body timeContainer="par">
        <div xml:lang="und">
            <p xml:id="caption_7236917318434_0" region="0_caption_7236917318434_0" xml:space="preserve" begin="22336:09:52.427" end="22336:09:53.987">
                <span style="s000">(Musique douce)</span>
            </p>
            <p xml:id="caption_7236917473234_0" region="0_caption_7236917473234_0" xml:space="preserve" begin="22336:09:54.147" end="22336:09:54.287">
                <span style="s001">Mama...</span>
            </p>
        </div>
    </body>
</tt>

icbaker commented 8 months ago

I assume the problematic parts of your example are this?

tts:color="rgba(255,0,254,1.00)" tts:backgroundColor="rgba(0,0,0,0.50)"/>
tts:color="rgba(255,255,255,1.00)" tts:backgroundColor="rgba(0,0,0,0.50)"/>

tts:color and tts:backgroundColor are both defined as 'type' <color> as defined in section 10.3.9 linked above. As I quoted, the alpha range for these strings is 0 (transparent) to 255 (fully opaque). So the strings in your example define colors that are all very transparent (1.00 means 0.39% opaque). Is that what you expect?

Xlcghs commented 8 months ago

Actually, shakaPlayer accepts float alpha values in rgba, unlike Exo which only accepts integers between 0 and 255 to achieve the desired transparency. Upon analyzing the two versions of the ttml standard (v1 vs v2), it was found that in ttml v1:

<color>
  : "#" rrggbb  | "#" rrggbbaa  | "rgb(" r-value "," g-value "," b-value ")"  | "rgba(" r-value "," g-value "," b-value "," a-value ")"  | <namedColor>

rrggbb
: <hexDigit>{6}

rrggbbaa
: <hexDigit>{8}

r-value | g-value | b-value | a-value
: component-value

component-value
: non-negative-integer // valid range: [0,255]

non-negative-integer
: <digit>+

In ttml v2:

<alpha>
  : float

So, my question is: does Exo support only the ttml v1 version? If not, is it possible to force it to support float alpha values without changing the Exo library? Or do I have to use exact colors instead of using the rgba format?

Thank you very much for your help.

icbaker commented 8 months ago

My reading of the TTML v1 spec suggests that it also expects rgba(r, g, b, a) where a is between 0 and 255. From section 8.3.2 that defines <color>:

A expression is used to specify a named color, exact RGB color triple, or exact RGBA color tuple, where the alpha component, if expressed, is maximum (255) at 100% opacity and minimum (0) at 0% opacity, and where the applicable color space is defined by [SRGB].

And the tts:color attribute is type <color>. ExoPlayer's logic (of expecting an alpha value between 0 and 255 in an rgba(r, g, b, a) expression) seems to be compatible with both TTML v1 and v2.

In ttml v2:
<alpha>
  : float

I'm not sure which part of the TTML v2 spec you're quoting here, but if it's 10.3.1, then I already addressed above in https://github.com/androidx/media/issues/1127#issuecomment-1966720279 that this is not relevant for the interpretation the tts:color and tts:backgroundColor attributes (since these are of type <color> not <alpha>).

Both TTML specs seem to be very clear to me. Shaka player is free to treat rgba(255, 0, 255, 1.0) as 'fully opaque', but this is not the expected behaviour from the spec, and it seems very risky for content producers to rely on this undefined behaviour (since it is unlikely to work consistently across players, as you're finding). It would be preferable for the content to obey the spec (that's sort of why the spec exists :)).

If not, is it possible to force it to support float alpha values without changing the Exo library?

You can either provide a custom SubtitleParser implementation for TTML (likely inspired by forking TtmlParser), and configure the player with your custom SubtitleParser.Factory to construct it. Or you can fork the library, depend on it locally, and change just the parseTtmlColor line you highlighted above in your original comment.

Xlcghs commented 8 months ago

Thank you for your feedback

androidx / media

TTML subtitles - Opacity not compliant with TTML specification #1127