search

w3c / ttml2

Timed Text Markup Language 2 (TTML2)
https://w3c.github.io/ttml2/
Other
41 stars 16 forks source link

Audio - gain and mute suggestions. #1235

Open btsimonh opened 3 years ago

btsimonh commented 3 years ago

Hi all,

after discussion with Nigel, I'm adding a quick note in here as a possible user/implementer of TTML2 for audio.

Yella Umbrella would like to have tts:gain not restricted to -1 through 1, as we record audio, and allow the users to add gain if the recorded audio is too low.

We would also like to suggest a tta:mute attribute on the audio element which would tell presentation engines that they can effectively ignore the source (not retrieve it, save bandwidth). The reasoning being that during authoring you may have multiple takes which the user may wish to retain, but are of no use in the final presentation. We could use tts:gain=0, but feel something more explicit is appropriate for real implementations (and also - we wish to retain the gain parameter for the take for future possible use!).

br,

Simon Hailes

nigelmegitt commented 3 years ago

It's easy to see how abs(gain) > 1 would work, and should be an easy specification change to make.

However it's more complex to understand how tta:mute would work. Is it like tts:display="none", i.e. the element on which it is specified, and all of its descendants, do not contribute to the audio output?

btsimonh commented 3 years ago

Yes, tta:mute would tell the presentation engine that that element and children can be ignored - 'are not intended for presentation'.
I would suggest that to keep things simple, it's not animatable. Whether that means that they could be pruned during ISD creation, and what side effects that may have, I don't know. One question would be do we restrict it to audio elements, or allow it on any content element. For the immediate use case, I would be happy with it being only on audio elements - that it solves a number of possible issues ('do I prune the span containing text because it is mute?'). Note that 'not animatable' actually comes from 'element and children ignored', although I would specify explicitly as well.

example of mute:

      <p xml:id="ad31b" begin="30s" end="40s">
        <animate begin="0.0s" end="0.12s" tta:gain="1;0.39"/>
        <animate begin="9.8s" end="10s" tta:gain="0.39;1"/>
        <span>
          <audio src="DRAD182Y01.wav" clipBegin="35.68s" clipEnd="37.72s" begin="0.12s" end="2.16s"/>
          <audio src="DRAD182Y01.wav" clipBegin="35.68s" clipEnd="37.72s" begin="2.1s" end="5s"/>
          <audio src="DRAD182Y02.wav" begin="0.13s" end="5s" tta:mute="1"/>
          <audio src="DRAD182Y03.wav" begin="0.5s" end="3.5s" tta:mute="1"/>
          Nick takes a drag of his cigarette.</span>
      </p>

DRAD182Y02.wav and DRAD182Y03.wav do not need to be retrieved (for this p), and will not be played. (note I used "1" rather than "true" to shorten the XML. For me, it's boolean.)

Note: I don't think tta:mute is essential for ttml2. But I do believe it is a useful addition to get in whilst audio implementations are few.