search

video-dev / hls.js

HLS.js is a JavaScript library that plays HLS in browsers with support for MSE.
https://hlsjs.video-dev.org/demo
Other
14.42k stars 2.55k forks source link

SUPPLEMENTAL-CODECS attribute support (DolbyVision 8.4) #5558

Open legosoff opened 1 year ago

legosoff commented 1 year ago

Is your feature request related to a problem? Please describe.

Good afternoon! You will have to do whether HLS supports SUPPLEMENTAL-CODECS=?

Describe the solution you'd like

EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=24398447,BANDWIDTH=57691258,VIDEO-RANGE=PQ,CODECS="hvc1.2.4.H150.b0",RESOLUTION=3840x1608,FRAME-RATE=23.976

,SUPPLEMENTAL-CODECS="dvh1.08.06" video/hvc1/2/playlist.m3u8

ID : 1
Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : Main 10@L5@High
HDR format : Dolby Vision, Version 1.0, dvhe.08.06, BL+RPU, HDR10 compatible / SMPTE ST 2094 App 4, Version 1, HDR10+ Profile B compatible
Codec ID : hvc1
Codec ID/Info : High Efficiency Video Coding
Duration : 2 ч. 8 м.
Bit rate : 24,4 Мбит/сек
Width : 3 840 пикселей
Height : 1 608 пикселей
Display aspect ratio : 2,40:1
Frame rate mode : Постоянный
Frame rate : 23,976 (23976/1000) кадра/сек
Color space : YUV
Chroma subsampling : 4:2:0 (Type 2)
Bit depth : 10 бит
Bits/(Pixel*Frame) : 0.165
Stream size : 21,8 Гбайт (100%)
Color range : Limited
Color primaries : BT.2020
Transfer characteristics : PQ
Matrix coefficients : BT.2020 non-constant
Mastering display color primaries : Display P3
Mastering display luminance : min: 0.0050 cd/m2, max: 1000 cd/m2
Maximum Content Light Level : 820 cd/m2
Maximum Frame-Average Light Level : 330 cd/m2
Codec configuration box : hvcC+dvvC

Which those who have supports dolby vision and now automatic transmission to SUPPLEMENTAL, and who does not, the transmission will usually be HDR, and HDR10+.

Additional context

https://developer.apple.com/documentation/http-live-streaming/hls-authoring-specification-for-apple-devices-appendixes

robwalch commented 1 year ago

Hi @legosoff,

Do you have an sample HLS assets with DolbyVision 8.4 which requires SUPPLEMENTAL-CODECS available for testing?

legosoff commented 1 year ago

Hi @robwalch. I can do it, come on.

legosoff commented 12 months ago

I have version 1.4.7, Dolby Vision 8.4 HLG WebOS and iOS does not work.

legosoff commented 12 months ago

When it's ready version 1.5.0? Very long, I need Dolby Vision. videojs-contrib-hls.js it works for him Dolby Vision 8.1, 8,4 and 5 excellent, your hls.js late.

robwalch commented 11 months ago

What do you mean by "does not work"? This is a feature request. If you have an HLS asset and use-case that is not working, file a bug.

legosoff commented 11 months ago

How do I know to report a bug? I have so did everything right. VideoJS HLS i have Dolby Vision running.

Video

EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=53145459,BANDWIDTH=51558427,VIDEO-RANGE=HLG,CODECS="hvc1.2.4.H153.b0",RESOLUTION=3840x2160,FRAME-RATE=60.000

,SUPPLEMENTAL-CODECS="dvh1.08.09" video/hvc1/playlist.m3u8

I-Frame Playlists

EXT-X-I-FRAME-STREAM-INF:AVERAGE-BANDWIDTH=1176864,BANDWIDTH=1208040,VIDEO-RANGE=HLG,CODECS="hvc1.2.4.H153.b0",RESOLUTION=3840x2160,URI="video/hvc1/fragment.m3u8"

,SUPPLEMENTAL-CODECS="dvh1.08.09"

Complete name : C:\temp\hevc.mp4
Format : MPEG-4
Format profile : Base Media / Version 2
Codec ID : mp42 (isom/mp42/hvc1/dby1/iso5)
File size : 28.4 MiB
Duration : 4 s 483 ms
Overall bit rate : 53.2 Mb/s
Frame rate : 60.000 FPS
Encoded date : 2023-07-12 14:30:29 UTC
Tagged date : 2023-07-12 14:30:29 UTC

Video

ID : 1
Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : Main 10@L5.1@High
HDR format : Dolby Vision, Version 1.0, dvhe.08.09, BL+RPU, HLG compatible
Codec ID : hvc1
Codec ID/Info : High Efficiency Video Coding
Duration : 4 s 483 ms
Bit rate : 53.1 Mb/s
Width : 3 840 pixels
Height : 2 160 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 60.000 FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 10 bits
Bits/(Pixel*Frame) : 0.107
Stream size : 28.4 MiB (100%)
Color range : Limited
Color primaries : BT.2020
Transfer characteristics : HLG
Matrix coefficients : BT.2020 non-constant
Codec configuration box : hvcC+dvvC

WebOS and iPhone 11 before 14 does not work.

robwalch commented 11 months ago

WebOS and iPhone 11 before 14 does not work.

iPhone with iOS 16.x does not support MSE which is required by HLS.js. Have you tried Safari desktop and iPad?

video.js may allow the native player to handle HLS playback with <video src=m3u8/>. HLS.js only plays HLS through MSE. To make it work, follow the embedding guidelines in the README:

https://github.com/video-dev/hls.js/blob/master/README.md#embedding-hlsjs

For WebOS (LG tvs?) the Web browser behavior is similar to Chrome, but it appears that the SourceBuffers need to be instantiated with the a DolbyVision codec rather than HEVC at least in the TV browser, as attempting to play DV 8.1 or 8.4 with MSE produces the error: CHUNK_DEMUXER_ERROR_APPEND_FAILED: Video stream codec dolbyvision doesn't match SourceBuffer codecs.. That being said MediaSource.isTypeSupported support checks for HEVC pass, but DV codec support checks fail, so it is unclear if the Web browser on WebOS (I'm testing on an LG CX) allows DV playback through MSE.

Please rather than stating 'does not work", provide an example, steps to reproduce, and actual/expected behavior as outlined in the Bug Report Template as part of your issue.

legosoff commented 11 months ago

Hi, robwalch! Thanks for the reply. I have an LG 1C webOS. I turn on native streams mode, Dolby Vision works, but the download is very bad and delayed, the subtitles don't work in manifests. I'm waiting for its version 1.5.0, let's see.

robwalch commented 11 months ago

Updated 11/28/23:

Moving to v1.6.0. There is no codec support check that uses the value of SUPPLEMENTAL-CODECS. In theory, the values can be mapped to a MediaCapabilities hdrMetadataType value to determine if the DV metadata will be used. Unfortunately regardless of the MediaCapabilities implementation and result, the primary HEVC codec still needs to pass isTypeSupported checks. If it does, playback should be supported.

robwalch commented 9 months ago

See https://developer.apple.com/documentation/http-live-streaming/hls-authoring-specification-for-apple-devices-appendixes#The-SUPPLEMENTAL-CODECS-attribute

Codec CODECS attribute SUPPLEMENTAL-CODECS attribute VIDEO-RANGE attribute
Dolby Vision 8.4 hvc1.2.4.L153.b0 dvh1.08.07/db4h HLG
Dolby Vision 8.1 hvc1.2.4.L150 dvh1.08.06/db1p PQ

As long as a device supports the CODECS attribute ("hvc1.2.4.L153.b0" or "hvc1.2.4.L150") then playback should consist of at least HLG or HDR10 color profile information. An advanced MediaCapabilities implementation is required to test for supplemental support of Dolby Vision 8.1/4 hdrMetadatType on top of these HEVC checks as some players will only report that they support DoVi profile 5 outright.

avelad commented 7 months ago

It seems that Chromium is working on supporting the case talked about here, see: https://chromium-review.googlesource.com/c/chromium/src/+/5058372 (Video: Support clear Dolby Vision downgrade playback for MSE)

StaZhu commented 7 months ago

For hdrMetadatType, since ST2094-10 !== Dolby Vision RPU NALU, so chrome will always return false for this typeof metadata, just don't use it test DV support!

For that MSE error: CHUNK_DEMUXER_ERROR_APPEND_FAILED: Video stream codec dolbyvision doesn't match SourceBuffer codecs., this should be fixed if Chromium > 122.0.6168.0 and DV profile is cross compatible, its working on Chrome Canary 122 now! you will get a MediaLog if append sourceBuffer with hevc codec instead of dolby vison:

Dolby Vision video track with track_id=1 is using cross-compatible codec: hevc. To prevent this, where Dolby Vision is supported, use a Dolby Vision codec string when constructing the SourceBuffer.

Remember this MediaLog doesn't mean Chrom will support DV. on most of the platform, DV is not supported.

As my CL described, for web developers, the logic will be something like below:

if (isTypeSupported('video/mp4;codecs="dvh1.08.07"')) {
  if (use_rpu) {
    // Should be success and rendered with RPU meta.
    source.addSourceBuffer('video/mp4;codecs="dvh1.08.07"');
    ...
  } else if (dvcc.dv_bl_signal_compatibility_id === 1 ||
             dvcc.dv_bl_signal_compatibility_id === 2 ||
             dvcc.dv_bl_signal_compatibility_id === 4) {
    // Should be success and rendered without RPU meta as
    // normal HLG/HDR10/SDR contents.
    source.addSourceBuffer('video/mp4;codecs="hev1.2.4.L120.90"');
    ...
  } else {
    // Downgrade is not allowed, playback fails.
  }
} else if (isTypeSupported('video/mp4;codecs="hev1.2.4.L120.90"')) {
  if (dvcc.dv_bl_signal_compatibility_id === 1 ||
      dvcc.dv_bl_signal_compatibility_id === 2 ||
      dvcc.dv_bl_signal_compatibility_id === 4) {
    // Should be success and rendered without RPU meta as
    // normal HLG/HDR10/SDR contents.
    source.addSourceBuffer('video/mp4;codecs="hev1.2.4.L120.90"');
    ...
  } else {
    // Downgrade is not allowed, playback fails.
  }
} else {
  // HEVC is not supported, playback fails.
}

Just have a try :)

robwalch commented 7 months ago

Hi @StaZhu.

HLS.js creates SourceBuffers based on the Variant CODECS or Initialization Segment (MAP) stsd box parsed codec (v1.5+). SUPPLEMENTAL-CODECS should not be used for video/mp4;codecs=".

Can you provide a little more information on which devices and user-agents would require the workaround above, and where you would expect to get the values or use_rpu and dvcc? HLS.js currently does not parse mp4 dvcC boxes.

StaZhu commented 7 months ago

Hi @StaZhu.

HLS.js creates SourceBuffers based on the Variant CODECS or Initialization Segment (MAP) stsd box parsed codec (v1.5+). SUPPLEMENTAL-CODECS should not be used for video/mp4;codecs=".

Can you provide a little more information on which devices and user-agents would require the workaround above, and where you would expect to get the values or use_rpu and dvcc? HLS.js currently does not parse mp4 dvcC boxes.

Summary:

For profile8, use hevc mimetype construct source buffer. (for Chrome 110 ~ 121 on Windows platform, the playback will still fail because of the CHUNK_DEMUXER_ERROR_APPEND_FAILED error you guys mentioned).

For profile5, use dolby mimetype construct source buffer.

More information can be found here: https://docs.google.com/document/d/1pZL21Q_Oi3wkNMuw1kfMlea2HuZ34o954uP34U_Xec4/edit#heading=h.6ipacul7sis6

legosoff commented 6 months ago

OLED Konka A55 WebOS hevc through MSE works. Hlsjs hevc does not work.