search

cta-wave / dash-hls

For work on the DASH-HLS Interoperability specification
10 stars 0 forks source link

DASH track roles and HLS characteristics #40

Closed RufaelDev closed 1 year ago

RufaelDev commented 2 years ago

It would be helpful to have a mapping between DASH roles and HLS characteristics and vice versa, this would help in converting DASH to HLS and vice versa.

RufaelDev commented 2 years ago

https://dev.w3.org/html5/html-sourcing-inband-tracks/#mpegdash might be another useful reference

technogeek00 commented 2 years ago

Prioritization Questionaire

Question Answer
Does the feature relate to an industry streaming use-case? Yes, Track Role Identification
- What is the commonality of this case? Very Common
- Is this an established or emerging practice? Established
Does this feature have related mechanisms in both DASH and HLS? Yes, Different Schemes
- What is the maturity of support in both specifications? DASH - Mature, HLS - Mature
- What is the maturity of implementation support for both specifications? DASH - Immature, HLS - Mature
- Are there known interoperability issues between specs? Yes, scheme conversions not specified, DASH implementations not clarified
- Has anyone implemented this interoperably? No
- Are there features missing in a specification with open proposals for it? No
Has the industry defined defacto mechanisms not present in both DASH and/or HLS? No
- Why was functionality defined outside of the main specifications? N/A
- Has the functionality been standardized elsewhere (DASH-IF, CTA, SVA)? N/A
- Is the functionality proprietary or openly developed? N/A
- Could the functionality be incorporated into specifications with evangelism? N/A

Working Notes

technogeek00 commented 1 year ago

Box Conformance Notes

Manifest Converstion Notes

Role@value Media Types Descriptions
caption video, text Captions
subtitle video, text Subtitles
main any Main media for use if no other information provided
alternate any Alternative to main media components based on other accompanying descriptors
supplementary any Additive media to main media but of a different type
commentary audio, text Commentary media
dub audio, text Media containing a different language than the original
description audio, text Textual or audio description of a visual component
sign video Visual sign-language interpretation of an audio component
metadata text, application Information intended to be processed by application specific elements
enhanced-audio-intelligibility audio Improved intelligibility of the dialog
emergency any Information about a current emergency
forced-subtitle text Textual information for display when no other text representation is selected
easyreader text, video Simplified or reduced caption track
karaoke any Textual representation of a songs lyrics
Characteristic Media Types Descriptions
public.subtitles.forced-only text Track contains only forced subtitles
public.accessibility.describes-music-and-sound text Transcription including description of auditory cues
public.accessibility.describes-video audio Audible content describing the visual component
public.translation.dubbed audio, text Media containing a different language than the original
public.easy-to-read text Simplified or reduced caption track
public.auxiliary-content any Content auxilary to the main content
public.main-program-content any Main media essential to the presentation
public.accessibility.transcribes-spoken-dialog text Transcription including spoken dialog
DASH Role HLS Characteristic Notes
caption public.accessibility.transcribes-spoken-dialog
subtitles n/a Implicit from TYPE=SUBTITLES
main public.main-program-content Generally non-included descriptions for both specs
alternate n/a Implicitly signalled by other characteristics being present
supplementary public.auxiliary-content Generally aligned with additional roles / characteristics / attributes describing details
commentary public.auxiliary-content Explicit commentary signal not available in HLS, but NAME attribute can aid identification to users
dub public.translation.dubbed
description public.accessibility.describes-music-and-sound, public.accessibility.describes-video HLS characteristics may be used together or independently
sign n/a No equivalent
metadata n/a Not generally translatable
enhanced-audio-intelligibility public.accessibility.enhances-speech-intelligibility HLS value proposed by interest group, but not officially in support
emergency n/a No equivalent
forced-subtitled | n/a | Implicit fromFORCED=YES`
easyreader public.easy-to-read
karaoke n/a No equivalent
Yasser-Comcast commented 1 year ago

So two things: 1) What about DASH Accessibility values if used? 2)How would one indicate "alternate commentary" in HLS? Maybe it might be useful listing that out.

RufaelDev commented 1 year ago

This is excellent start, thanks so much for bringing this forward. To add to yasser's point.

DVB uses Accessibility element with tv anytime descriptors to add information for alternate, and commentary roles with audio, it may be useful to use that information when deriving corresponding HLS representation. This may be a comment to complement the usage of commentary and alternate roles when using audio schemeIdUri is urn:tva:metadata:cs:AudioPurposeCS:2007 @value=1 -> visually impaired Role is commentary or alternate @value=2 for hard of hearing Role is alternate

Another usage is for using roles in text, this Accessibility can be used to signal hard of hearing as well with Role=main. This is something we commonly support when outputting DASH with HoH.

It may be interesting to add some of this information to the mapping.

RufaelDev commented 1 year ago

So basically in some cases different track roles for dash can map to public.accessibility.transcribes-spoken-dialog,public.accessibility.describes-music-and-sound, such as a timed text track with role main and an accessiblity descriptor indicating hard of hearing.

also so minor editorial fixes forced-subtitle