ingararntzen
commented
11 months ago Hi @AdamSobieski
Please be aware there is another approach, so that video composition could be achieved without extensive changes to the media player. Similarly - the reflowability could be accomplished without changes to the distribution protocol.
The timing object proposal was also motivated by very similar use cases. The proposal is a bit old now, but key idea is remains relevant:
- avoid changes to distribution protocols (like dash)
- avoid turning the media player into a studio
- avoid limiting client-side composition to video content - for instance also support layers of timed html-based graphics
- avoid limiting client-side composition to single content source or single distribution technology
The key change required by the timing object proposal would be that media players were able to sync to a timing object. The Vimeo player now supports this, I think. When sync is supported, the remaining challenges are less daunting.
AdamSobieski
nigelmegitt
In addition to sets of video tracks in containers and streaming solutions representing alternative renditions of content, let us consider describing sets of video tracks where those tracks are intended to be blended together by clients per alpha compositing.
Alpha composting is, today, most often performed server-side before content is streamed or broadcast by video production software.
Uses of client-side video layer compositing would include:
Some video layers could be delivered with a variable bitrate, e.g., news anchors speaking, while other video layers could be delivered with an assured fidelity, e.g., text overlays, lower third content, and news tickers.
Video players could automatically select which video layers to composite together on behalf of end-users. A client-side system could, for instance, automatically select a foreground layer from a set of alternatives based on a user's preferred language.
Web pages could allow end-users to make use of menus to dynamically select which video layers to composite together. End-users could, for instance, select which variety of news that they would prefer for a news ticker composited atop other video content.
Client-side compositable video layers could be a part of solutions for enabling interactive time-based hypermedia, e.g., hypervideo. Foreground layers could be capable of having hypertext selected from them or of otherwise being interacted with, e.g., clicked upon. As considered, this type of video layer might provide bounding boxes or silhouettes, perhaps with identifiers unique to the video container or stream.
A useful implementation concept could be that of "trees of video tracks". With such trees, we could express sets of video tracks for alpha compositing where some of these layers could have alternative renditions or reflowed renditions. Recursively, some of these layers could be intended to be rendered using client-side video layer compositing.
Perhaps, one day, video containers and streaming technologies will be able to combine client-side compositability with reflowability to provide video producers and consumers with new features and functionalities.
Thank you. I look forward to discussing these ideas with you.