Workshop on Bundling Interactive Media Content on the Web

[tidoust]

I'm exploring the possibility to organize a workshop on bundling interactive media content on the Web later this year (mid year?), following initial exchanges with media industries in that space.

See draft Call for Participation for a possible scope and content. This is still early stage, but could make quick progress. Focus on media does not exclude a broader discussion on bundling for other purposes if people are interested.

So far, the list of topics does not include specific entries for horizontal activities. What would be worth highlighting in a bundling discussion?

[r12a]

It's not clear to me how much this workshop will get in to useability and user interfaces. For example we have a question at the moment about whether audio controls can be rearranged and reshaped to fit in vertical writing mode, and there are probably other questions to ask wrt RTL scripts.

[tidoust]

@r12a I think it's unlikely that the workshop would get in user interfaces per se, but discussions on manifest files are never far from discussions on packaging, and no doubt that triggers internationalization requirements (without going very far, I note that the WebAppManifest dictionary starts with a dir and lang members...).

[svgeesus]

Is this workshop still intending to be f2f, or virtual?

[tidoust]

First question I was trying to answer was: should this workshop take place? The answer to that question seems positive. How to run the workshop is what I'm now trying to figure out. A physical workshop seems hard to envision in the short- to mid-term.

[himorin]

No request on review from i18n WG. One comment raised in discussion, on another topic to add to the topics section:

• Alternative strategies for bundling interactive media content in areas, devices or social contexts (like Coronavirus) where bandwidth is restricted.

[samuelweiler]

How can you better distinguish this from the work already in progress, specifically in the IETF WPACK WG? The fourth bullet in the draft CFP seems particularly duplicative of work in WPACK:

Do possible bundling solutions play nicely with edge cache servers? For instance, do they allow to distribute content from edge cache servers in a way that preserves Web security guarantees (based on the notion of origin), and avoids duplication of content?

Let me know if you would like an introduction to the WPACK chairs.

My instinct is that some and perhaps all of these use cases could be satisfied with existing tech + WPACK. If we need a workshop, can you be even more clear about what pieces are missing? Or do we instead need a hackathon to try to build these things using existing tech (+WPACK)?

BTW, I find the DVD use case in the draft CFP very relatable, but it's also one that I can imagine being built with existing tech.

[dwsinger]

Having Bert talk about web/media sync and how he did it for these slides would be nice (it's one of the web/media integration high nails).

[tidoust]

@samuelweiler said:

How can you better distinguish this from the work already in progress, specifically in the IETF WPACK WG? The fourth bullet in the draft CFP seems particularly duplicative of work in WPACK

I added a reference to the IETF WPACK group (it did not formally exist last time I touched the draft) and to the Content-Based Origins for the Web draft. The goal of the workshop is not to design a solution from scratch if existing or ongoing ones satisfy needs. It is to evaluate whether that work meets interactive media bundling needs (and clarify these needs to start with) and encourage the media industry to converge on it if it does (or gauge whether it could be tweaked to meet media needs if not).

For instance, my understanding from exchanges with Jeffrey Yasskin some time ago is that Web Bundles is not necessarily being designed with "dynamic" creation of bundle use cases in mind, and might not be a perfect fit for media streaming scenarios where you need to assemble resources on the fly (to select audio/video tracks in the right language, resolution, codecs, etc.). Happy to talk with WPACK chairs, I would certainly like WPACK folks to be involved in the discussions and represented on the program committee.

My instinct is that some and perhaps all of these use cases could be satisfied with existing tech + WPACK. If we need a workshop, can you be even more clear about what pieces are missing? Or do we instead need a hackathon to try to build these things using existing tech (+WPACK)?

Use cases may all be satisfied with existing tech + WPACK. Even if they do, there needs to be general agreement that this is the case across the media industry, and general adoption of these solutions. In particular, I'd like to see how this approach can align with the approach based on media containers that the media industry is used to following, c.f. work on CMAF recently.

BTW, I find the DVD use case in the draft CFP very relatable, but it's also one that I can imagine being built being existing tech.

This may indeed seem to boil down to "package your web app and you're done". And perhaps that's all it takes. There are additional dimensions that are captured in other questions, including:

• The media industry continues to develop dedicated technologies to glue bits together, for instance specific manifest file formats to convey semantics. It may be that this is all fine and a media-industry-only problem. Or it may be that there is an opportunity to align efforts with other industries such as digital publishing and "regular" web apps. See TAG's discussion on manifests.
• Media content providers need to prepare and distribute content across different distribution channels (live streaming, on-demand streaming, broadcasting, offline distribution) and different consumption experiences, some more directly "web" than others: the app runs on its own within a browser, the app runs within a media player app of some kind owned by a third party, or the app uses web technologies but runs on a media device that follows an audio/video first approach (e.g. a TV). Can one solution rule them all?
• As mentioned above, the starting point in the media industry is often the media container. Among other things, it is not entirely clear how the creation of an app impacts the media production pipeline that currently revolves around media containers. Said differently, what is the incremental story?

[tidoust]

@himorin said:

One comment raised in discussion, on another topic to add to the topics section: Alternative strategies for bundling interactive media content in areas, devices or social contexts (like Coronavirus) where bandwidth is restricted.

Do you know whom could speak about the issue and how it could affect possible solutions? That is, the need to adapt to available network bandwidth constraints is at the core of media streaming solutions and by extension to solutions that could be used for bundling media for distribution over networks, so I'm wondering what else would need to be considered.

[dwsinger]

You have the possibility of format(s) based on the HEIF/MP4 family, I assume?

[dwsinger]

I think the workshop should cover interoperation as well as packaging, by the way (notably sync'd web and media content). There are also ad-insertion questions...

[tidoust]

@dwsinger said:

You have the possibility of format(s) based on the HEIF/MP4 family, I assume?

HEIF/MP4 fit in the media container formats I was alluding to in my response to @samuelweiler but I'm not sure I understand the context for this comment. Could you clarify?

I think the workshop should cover interoperation as well as packaging, by the way (notably sync'd web and media content). There are also ad-insertion questions...

Do the following two questions in the Which topics will be covered section correctly capture these needs, or do you see something missing?

• How to orchestrate synchronized playback of media and non-media content, and guarantee consistent user experiences (UX) across devices? For example, how to make a tutorial-like experience, where a lecture is presented, in context, in a web experience, with slides and diagrams delivered using web technologies but synchronized to the video?
• More generally, how to manage the insertion and overlay of ancillary and interstitial content, including but not limited to advertisements, logo overlays, explanatory overlays, and the like?

[dwsinger]

that covers it...