search

w3c / tt-reqs

Timed Text requirements
https://w3c.github.io/tt-reqs/ttml2-2e-reqs/
Other
3 stars 5 forks source link

Support for Responsive Timed Text #10

Open cconcolato opened 5 years ago

cconcolato commented 5 years ago
  1. Context

Netflix produces different versions of the same video content with various resolutions and to be displayed on devices with different aspect ratios: portrait aspect ratio (TV, desktop, mobile ) and landscape aspect ratio (mobile devices). Timed text authored for a landscape aspect ratio does not appear correctly when displayed on a portrait aspect ratio device.

Some problems are:

Responsive Web Design is a concept that is common to solve this problem for web pages. Is it based on CSS Media Queries, with dynamic layouts and break points. A similar approach should be applied to TTML and IMSC.

Example: 16x9 reference rendering 16x9-reference

Example: 9x16 rendering issues

  1. Requirements

R1: It shall be possible to author a single timed text document that adapts to a variety of screen resolutions, aspect ratios, and readability criteria (e.g. language).

R2: It shall be possible for an author to indicate opportunities for splitting an event into multiple events based on presentation context.

  1. Possible solutions

3.1 Condition and Media Queries The current problem could be solved as follows, one problem being that there are 2 elements with the same xml:id.

<tt …> <!-- the document explicitly does not mention displayAspectRatio.--> 
  <head>
    <styling>
    <!-- two style elements with the same ID are used 
           with non-overlapping conditions (aspect ratio)-->
      <style tts:fontSize="5%" ... xml:id="span" 
             condition="media('(device-aspect-ratio:16/9)')"/>
      <style tts:fontSize="2%" ... xml:id="span"
             condition="media('(device-aspect-ratio:9/16)')"/>
    </styling>
    <layout>
    <!-- two region elements with the same ID are used 
           with non-overlapping conditions-->
      <region tts:extent="80.00% 80.00%" tts:origin="10.00% 10.00%"
              xml:id="bottom" 
              condition="media('(device-aspect-ratio:16/9)')"/>
      <region tts:extent="90.00% 70.00%" tts:origin="5.00% 5.00%"
              xml:id="bottom" 
              condition="media('(device-aspect-ratio:9/16)')"/>
    </layout>
  </head>
<!-- the use of region or style attribute is the same as if there would be only one region or style element -->
  <body region="bottom">
    <div>
      <p begin="0s" end="3s" style="span">I know he's Winston Churchill and all that, but remember who you are.</p>
    </div>
  </body>
</tt>
  1. Splitting events

Another complementary option would be to add markers to the text on where the split of an event is acceptable, maybe with some conditions too:

<p begin="0s" end="3s" style="span">I know he's Winston Churchill and all that,<marker begin="1.5s" type=”split”/> but remember who you are.</p>

The result would look like:

nigelmegitt commented 5 years ago

BBC has similar use cases and I would support work to meet this requirement.

andreastai commented 5 years ago

@cconcolato As you point out

one problem being that there are 2 elements with the same xml:id

because the values of xml:id must be unique.

andreastai commented 5 years ago

In general it make sense to have media queries for timed text.

skynavga commented 5 years ago

@tairt Just a reminder, but TTML2 already supports media queries via the @condition attribute. So the question here (for me) is whether there is a barrier to using this existing support for this application (responsive content). There is also a related question about whether to support this TTML2 feature in IMSC.

nigelmegitt commented 5 years ago

@skynavga one obvious barrier is that the media queries that need to be supported are not listed in any form, so they aren't really usable in real world applications.

skynavga commented 5 years ago

@nigelmegitt well, since CSS (MQ itself) does not specify or make use of any mechanism to describe which MQs are supported, then TTML is no worse off;

My presumption about level of support is that, if an implementation supports #condition-fn-media, then it must support all syntactic expressions permitted by Media Queries; however, Media Queries does not specify any level of semantic support (or fall-back behavior) for processing MQs. This is something we could potentially address in TTML2 2e (as informative and/or substantive clarifications).

Regarding this specific proposal, it is my opinion that R1 is already supported (by TTML2 at least). However, R2 presents a problem in that we have defined conditions (in 8.2.1) so as to not affect timing decisions:

the timing semantics of a timed element are not ignored;

In order to support R2, I suspect we would need to add support for event based timing, such as found in [SMIL 3.0 5.6.3] (https://www.w3.org/TR/SMIL3/smil-timing.html#q135), a move that would certainly complicate our timing model.

css-meeting-bot commented 5 years ago

The Timed Text Working Group just discussed Support for Responsive Timed Text tt-reqs#10, and agreed to the following:

The full IRC log of that discussion <nigel> Topic: Support for Responsive Timed Text tt-reqs#10
<nigel> Peter: [has to leave]
<nigel> github: https://github.com/w3c/tt-reqs/issues/10
<nigel> Nigel: Cyril raised this and I supported it. Does anyone have anything bad to say about it right now?
<nigel> Pierre: I think this approach is worth a try.
<nigel> Glenn: My conclusion is requirement 1 is already supported and requirement 2 is problematic.
<nigel> Pierre: Problematic in the solution or the requirement?
<nigel> Glenn: I see it as an ask to support event based timing.
<nigel> Nigel: I don't see that.
<nigel> Cyril: I am not married to the marker proposal.
<nigel> .. The idea is to provide not line break opportunities but event based opportunities.
<nigel> .. It could be a conditional timing hierarchy for a given unit of content.
<nigel> Pierre: This is a really important topic. I don't know the answer, but if we succeed it will really help the
<nigel> .. industry so we should give it a shot.
<nigel> Glenn: One solution is to produce different documents.
<nigel> Pierre: There are multiple combinatory choices
<nigel> Glenn: This is also needed for karaoke where we have an algorithm for pushing content into a display
<nigel> .. region dynamically.
<nigel> .. You can't reuse IDs so that's off the table.
<nigel> Pierre: Setting aside the solutions, do you think the requirement is relevant?
<nigel> .. Looking at the images.
<nigel> .. Being responsive to those different aspect ratios.
<nigel> Glenn: I haven't given enough thought but it seems something worth considering.
<nigel> Andreas: From our side we think it is an important requirement.
<nigel> .. Making responsive web content is important and it would be good to do the same for TTML.
<Vlad> q+
<nigel> ack at
<nigel> Glenn: It is only the splitting that is an issue.
<nigel> ack vla
<nigel> Vlad: I agree it is important to be responsive in this way. I like the way you put it Cyril but the samples
<nigel> .. don't go beyond line breaks and text size, you don't do it justice, but the requirement is good.
<nigel> .. Responsive design and typography, i.e. using full scale variability using variable font support, which
<nigel> .. every browser does with CSS support and we are talking about scaling font from condensed to wide.
<nigel> RESOLUTION: We will attempt to meet these requirements in 2019.
gkatsev commented 5 years ago

I think the main downside of something like Media Queries is that it only applied to the device/browser itself. On the web, at least, you often have the player windows to a portion of the screen and it would be good if we can support that, though, could also happen at a later date like if/when element queries actually ship.

vlevantovsky commented 5 years ago

I support the requirements as proposed, but I also would like to note that the concept of "responsive design" goes well beyond the simple issues of text size scaleability and line breaking (as shown in illustrations). Variable fonts (already supported by CSS and all browsers) offer significant benefits here, allowing media queries being used to control text rendering by changing font properties such as width, weight, stylistic alternates, etc. So, I hope that the requirements themselves will be seen in a broader scope of "responsive timed text".