search

cta-wave / dash-hls

For work on the DASH-HLS Interoperability specification
10 stars 0 forks source link

Image-based trickplay tracks #37

Open technogeek00 opened 3 years ago

technogeek00 commented 3 years ago

Use Case Description

From @nicoweilelemental

For player resources optimization reasons, the use of a video track as a trickplay artefact is not always possible, and a lot of player providers recommend the use of image thumbnails tracks instead of special low framerate video tracks. DASHIF IOP section 6.2.6 covers this use case but there is equivalent support in the HLS RFC. There is the Image media playlists HLS extension proposal from Roku/Disney/Warner here https://developer.roku.com/docs/developer-program/media-playback/trick-mode/hls-and-dash.md but its relevance/adoption is limited by the fact that it's not part of the RFC. Same logic here: even if not supported by Apple players which don't need it as they can leverage iframe-only tracks, it would be super useful for the rest of the ecosystem to get this officially part of the HLS RFC.

Working Notes

Open Questions

Prioritization Questionaire

Updated post Apple proposal

Question Answer
Does the feature relate to an industry streaming use-case? Yes, Thumbnails / Trickplay
- What is the commonality of this case? Very Common
- Is this an established or emerging practice? Established
Does this feature have related mechanisms in both DASH and HLS? Yes
- What is the maturity of support in both specifications? DASH - Mature, HLS - Immature
- What is the maturity of implementation support for both specifications? DASH - Mature, HLS - Immature
- Are there known interoperability issues between specs? Yes, HLS proposal is net new with no existing content
- Has anyone implemented this interoperably? Yes, with non-spec aligned approach
- Are there features missing in a specification with open proposals for it? Yes
Has the industry defined defacto mechanisms not present in both DASH and/or HLS? Yes
- Why was functionality defined outside of the main specifications? Industry desire
- Has the functionality been standardized elsewhere (DASH-IF, CTA, SVA)? No
- Is the functionality proprietary or openly developed? Proprietary, but open
- Could the functionality be incorporated into specifications with evangelism? No

Original prior to Apple proposal

Question Answer
Does the feature relate to an industry streaming use-case? Yes, Thumbnails / Trickplay
- What is the commonality of this case? Very Common
- Is this an established or emerging practice? Established
Does this feature have related mechanisms in both DASH and HLS? No
- What is the maturity of support in both specifications? DASH - Mature, HLS - Missing
- What is the maturity of implementation support for both specifications? DASH - Mature, HLS - N/A
- Are there known interoperability issues between specs? Yes, HLS does not support
- Has anyone implemented this interoperably? Yes
- Are there features missing in a specification with open proposals for it? Yes
Has the industry defined defacto mechanisms not present in both DASH and/or HLS? Yes
- Why was functionality defined outside of the main specifications? Industry desire
- Has the functionality been standardized elsewhere (DASH-IF, CTA, SVA)? No
- Is the functionality proprietary or openly developed? Proprietary, but open
- Could the functionality be incorporated into specifications with evangelism? Yes
technogeek00 commented 3 years ago

Transfer of @nicoweilelemental comment:

Image-based trickplay tracks HLS image-based trickplay track was already a fairly common feature for VOD contents targeting the Roku platform. Now it's spreading in the live domain too, as the spec extension is simple and efficient, and there is a growing demand for it.

Here is an AWS Elemental stream showing the implementation of the Image Media Playlist spec for the live use case: https://d345o62q7f2dwl.cloudfront.net/referencestream/main.m3u8

It's fairly easy to configure with MediaLive + MediaStore, or MediaLive + S3: https://docs.aws.amazon.com/medialive/latest/ug/trick-play-roku.html

The implementation work is happening on MediaPackage too, as we speak, and in the player ecosystem. That would be really great to capitalize on the Roku/Disney/Warner spec, which is the de factor industry standard now.

technogeek00 commented 3 years ago

Additional note Disney Streaming's entire catalog will be aligned around this format.

technogeek00 commented 2 years ago

12/15/2021 Working Notes

HLS Notes

DASH Notes

Interoperability

Open Questions

nicoweilelemental commented 2 years ago

On 2/23, Apple announced upcoming iOS support for M-JPEG (ISO/IEC 23008-12 "High efficiency coding and media delivery in heterogeneous environments —Part 12: Image File Format") encapsulated in fMP4 (CMAF?) containers. Signaling example:

EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=331000,CODECS="mjpg",URI="iframe_index.m3u8"

M-JPEG prevents tiling but allows encryption. To be interoperable, it would need to be included in the DASH spec and supported by DASH players.

technogeek00 commented 2 years ago

Pulling back into research phase due to the Apple M-JPEG announcement, will discuss in the call tomorrow.

technogeek00 commented 2 years ago

Raised for broader discussion in CTA Spring Forum