Image profile should permit embedded images using smpte:image.

[skynavga]

At present, the image profile prohibits the use of smpte:image, forcing authors to always use external images. I believe this a bad long term strategy and unnecessarily restrictive.

[palemieux]

Moving to IMSC2 for consideration since there has been no active demand for such a feature in the scope of IMSC1 and TTML2 defines such capability.

[TobbeEdgeware]

I agree that this is too restrictive since it makes the subtitle document non-selfcontained. Especially when embedding IMSC1 image subtitles in MPEG-4 DASH segments, where the other media segments are not allowed to have references to external files. From an implementation point of view (I'm working with subtitling in dash.js), it is also easier to handle embedded images since the data flow is more similar to text-based subtitles, so this is what I implemented first.

[palemieux]

Especially when embedding IMSC1 image subtitles in MPEG-4 DASH segments where the other media segments are not allowed to have references to external files.

This is not accurate. ISO 14496-30:2014, which specifies the wrapping of TTML into ISOBMFF, specifically allows resources referenced by an XML document [to] be stored in the same subtitle sample as the document that references them, in which case they shall be stored contiguously following the XML document that references those resources, and provides specific guidance for PNGs (see Section 5.6).

Are you aware of any distribution or mastering format that stores images in-band the TTML file?

[TobbeEdgeware]

@palemieux Thanks for that reference, but I'd rather see that the example given in the SMPTE-TT specification would be allowed which embeds the image in the TTML itself.

The major spec of image subtitling used for Internet Streaming that I'm aware of is the Internet-Draft by Harmonic form 2012: https://tools.ietf.org/html/draft-smpte-id3-http-live-streaming-00.

It is for HLS and not fully compliant with the SMPTE-TT spec since it specifies tts:extent and tts:origin on the <div> instead of using a region reference, but is used by multiple vendors.

[palemieux]

@TobbeEdgeware I would not be excited to spend time on a feature (e.g. TTML-embedded PNGs) unless it is used in practice.

[TobbeEdgeware]

@palemieux I agree on that, but the HLS format I mentioned is used by some of our customers. A corresponding solution is needed for DASH. Typically the origin is a live feed with DVB subtitles (images) that needs to be converted for HLS and DASH.

[palemieux]

@TobbeEdgeware Good to know. Can you point these users to me and/or TTWG? I would like to better understand their interests and roadmap (since they are not using IMSC1 as we speak).

[nigelmegitt]

@TobbeEdgeware that internet draft expired in 2012 - do you happen to know if it was ever followed up, or if not, why not? It does not look from the IETF data tracker that anything was done, and indeed it was pulled from the rfc queue.

It also uses an unexplained syntax that has no meaning in TTML1 and is not referenced or explained, namely the use of tts:extent and tts:origin on elements other than region. We have attempted to add that into TTML2 but there is a question over exactly what is intended: do the extent and origin imply a new anonymous region (in which case what style attributes apply?), or do they imply that the currently active region is modified? In the second case existing content already on screen for that region could potentially be moved around when other content becomes active.

[TobbeEdgeware]

@nigelmegitt I don't think that Internet Draft was ever intended to be evolved into a standard. It is "Informational" and all IDs expire after 6months. It seems that the authors made some shortcuts compared to the TTML standard, so it is not obvious how to combine it with region elements. Still, it documents what Harmonic implements in its encoder and, as far as I understand, has become a de facto standard that others follow as well. There is a player "HLS Player +" in the app store https://itunes.apple.com/se/app/harmonic-hls-player-plus/id532114360?mt=8

[nigelmegitt]

Thanks @TobbeEdgeware. I've seen this flavour of output before, but I have never seen any explanation of how it is supposed to be interpreted. Just hoping to find someone who knows!

[skynavga]

On Fri, Aug 12, 2016 at 3:11 AM, Nigel Megitt notifications@github.com wrote:

@TobbeEdgeware https://github.com/TobbeEdgeware that internet draft expired in 2012 - do you happen to know if it was ever followed up, or if not, why not? It does not look from the IETF data tracker that anything was done, and indeed it was pulled from the rfc queue.

It also uses an unexplained syntax that has no meaning in TTML1 and is not referenced or explained, namely the use of tts:extent and tts:origin on elements other than region. We have attempted to add that into TTML2 but there is a question over exactly what is intended: do the extent and origin imply a new anonymous region (in which case what style attributes apply?), or do they imply that the currently active region is modified? In the second case existing content already on screen for that region could potentially be moved around when other content becomes active.

I have discussed this with another company who used this non-standard extensions, and they had not made a distinction between the two options you describe. In fact, they expressed surprise about the possibility of the second interpretation (since it may have unexpected side effects - moving around existing content). So effectively they assumed it meant create an anonymous region.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/w3c/imsc/issues/82#issuecomment-239398549, or mute the thread https://github.com/notifications/unsubscribe-auth/AAXCb6AhcO5vhv3PKR7JxZDOkAV8VGmFks5qfDitgaJpZM4Gd6kS .

[TobbeEdgeware]

@palemieux I looked a bit more at Section 5.6 of ISO 14996-30 and the possibility of appending the image data as subsamples in MP4 fragments. I think that is the best way to go for DASH, so we should try to make some example track for it. I'll add a note about it in the IMSC-1 Image Profile issue for dash.js and try to look at it later.

[palemieux]

@TobbeEdgeware It looks like GPAC supports the wrapping of TTML into ISOBMFF. I am not sure if it supports the wrapping of PNGs. Perhaps @cconcolato has some advice.

[mikedo]

There is a big difference between “wrapping of PNGs” in this context: 1) ISO BMFF wrapped TTML samples with PNG image subsamples (per ISO 14496-30); and 2) ISO BMFF wrapped TTML that has Base64 PNG encoding(s) using smpte:image inside a TTML document (as proposed by this issue).

From: Pierre-Anthony Lemieux [mailto:notifications@github.com] Sent: Tuesday, August 23, 2016 8:10 PM To: w3c/imsc imsc@noreply.github.com Subject: Re: [w3c/imsc] Image profile should permit embedded images using smpte:image. (#82)

It looks like GPAC supports https://gpac.wp.mines-telecom.fr/2014/09/04/subtitling-with-gpac/ the wrapping of TTML into ISOBMFF. I am not sure if it supports the wrapping of PNGs. Perhaps @cconcolato https://github.com/cconcolato has some advice.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/w3c/imsc/issues/82#issuecomment-241945694 , or mute the thread https://github.com/notifications/unsubscribe-auth/AEUs9CiF7Cp6GLNk-efRPkIF-OjZLUh7ks5qi7YQgaJpZM4Gd6kS . https://github.com/notifications/beacon/AEUs9MBftUURGgmS4ANcdEwv4vsRdtFJks5qi7YQgaJpZM4Gd6kS.gif

[palemieux]

@mikedo Yes. I meant (2) in response to https://github.com/w3c/imsc/issues/82#issuecomment-241871175 .

[cconcolato]

No, storage of images in TTML samples is not supported. 14496-30 has 2 methods: using a meta box or using subsamples. The one using the 'meta' box could be easily supported but the other one would need work.

[skynavga]

Are you saying that 14496-30 itself places restrictions on what is inside a TTML file? i.e., performs further/additional profiling? If not, then why would it matter whether TTML embeds an image itself? It is just TTML content at that point.

On Wed, Aug 31, 2016 at 3:14 AM, Cyril Concolato notifications@github.com wrote:

No, storage of images in TTML samples is not supported. 14496-30 has 2 methods: using a meta box or using subsamples. The one using the 'meta' box could be easily supported but the other one would need work.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/w3c/imsc/issues/82#issuecomment-243706084, or mute the thread https://github.com/notifications/unsubscribe-auth/AAXCb4QjlReFsAT0vePHV41VG0ec-WQJks5qlUYGgaJpZM4Gd6kS .

[cconcolato]

@skynavga To my knowledge, 14496-30 does not place restrictions on the TTML content. I was responding to @palemieux about the support in GPAC for the tools specified in 14496-30. But if the image is base64 embedded in the TTML file, then it should work as is with the current tools of GPAC.

[TobbeEdgeware]

The base-64 embedded image variant is not allowed by the IMSC-1.

I think it is great that 14996-30 has a more efficient and cleaner way to support embedded image subtitles using subsamples.

Unfortunately, the lack of support in GPAC (and other tools) and the lack of subsample information box support in the MP4 parser used in dash.js, leaves us with only base-64 encoded embedded image subtitles in DASH for the moment. I have implemented such support and its part of the upcoming 2.3.0 release of dash.js.

[skynavga]

On Wed, Aug 31, 2016 at 2:08 PM, Torbjörn Einarsson < notifications@github.com> wrote:

The base-64 embedded image variant is not allowed by the IMSC-1.

right, but this thread (and issue) is about allowing them in IMSC-2, which I believe is desirable (I would have preferred they be supported in IMSC-1)

I think it is great that 14996-30 has a more efficient and cleaner way to support embedded image subtitles using subsamples.

Unfortunately, the lack of support in GPAC (and other tools) and the lack of subsample information box support in the MP4 parser used in dash.js, leaves us with only base-64 encoded embedded image subtitles in DASH for the moment. I have implemented such support and its part of the upcoming 2.3.0 release of dash.js.

[palemieux]

Propose deferring this to imscVNext in absence of immediate industry need

[nigelmegitt]

I would also not include smpte:image, actually in any version of IMSC. If embedded images are required in IMSC 1.1 they should use the TTML2 mechanism for doing so.

[css-meeting-bot]

The Working Group just discussed Image profile should permit embedded images using smpte:image. imsc#82, and agreed to the following resolutions:

• RESOLUTION: WG agrees to put this on the backlog for a future version of IMSC.

The full IRC log of that discussion

<nigel> Topic: Image profile should permit embedded images using smpte:image. imsc#82
<nigel> github: https://github.com/w3c/imsc/issues/82
<nigel> RESOLUTION: WG agrees to put this on the backlog for a future version of IMSC.

[nigelmegitt]

(note that the absence of a milestone implies this is on the backlog)

[shobanaberlin]

Is there an update to this thread. There is a need to support DVB bitmap image in fragmented mpeg4 HLS. The only viable option seems to be embedding the image via IMSC. Is there an alternate option?

[palemieux]

@shobanaberlin Can you provide a link to the fragmented mpeg4 HLS specification? I am interested in understanding whether images can be carried within the wrapper/multiplex, instead of within the IMSC document.

[nigelmegitt]

@shobanaberlin @TobbeEdgeware @skynavga if you could add more detail about this requirement to w3c/tt-reqs#15, i.e. to the template text in that issue, if possible or if you don't have edit rights, by adding as a comment, that would be very helpful.

[palemieux]

@shobanaberlin Ping re: https://github.com/w3c/imsc/issues/82#issuecomment-452769921

[palemieux]

@shobanaberlin ISO/IEC 14496-30 specifies the carriage of images in MP4 (https://www.iso.org/standard/75394.html).