search

video-dev / hls.js

HLS.js is a JavaScript library that plays HLS in browsers with support for MSE.
https://hlsjs.video-dev.org/demo
Other
14.85k stars 2.58k forks source link

Inband Timed Metadata for FMP4 (ID3 within Emsgv1) #2360

Closed damienlevin closed 2 years ago

damienlevin commented 5 years ago

HLS JS should surface inband Timed Metadata for FMP4 (ID3 within Emsgv1).

The full AOM spec can be found here: https://aomediacodec.github.io/av1-id3/

Apple documentation: https://developer.apple.com/documentation/http_live_streaming/about_the_common_media_application_format_with_http_live_streaming

You can include one or more Event Message boxes (‘emsg’) in each segment. You must use version 1 of the Event Message box standard found in ISO-23009-1. The following values define the semantics:

scheme_id_uri Set to https://aomedia.org/emsg/ID3 to identify boxes that carry ID3v2 metadata.

value The URI that defines the semantics of the ID field. Any relative URI is considered to be relative to scheme_id_uri.

message_data Contains data compatible with ID3 version 2.x.x.

Happy to provide sample HLS playlists if needed.

johnBartos commented 5 years ago

@damienlevin Was made aware of this recently - thanks for filing an issue. Happy to try and squeeze this in for 1.0.0

damienlevin commented 5 years ago

That would be amazing, thanks !

johnBartos commented 5 years ago

A test stream would be great too!

On Fri, Aug 30, 2019 at 12:05 Damien Levin notifications@github.com wrote:

That would be amazing, thanks !

— You are receiving this because you were assigned.

Reply to this email directly, view it on GitHub https://github.com/video-dev/hls.js/issues/2360?email_source=notifications&email_token=ABGPAZDAQ6AQ3WA4KIZZAVLQHFAMHA5CNFSM4ISOTUH2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5SCQXI#issuecomment-526657629, or mute the thread https://github.com/notifications/unsubscribe-auth/ABGPAZA5I37CKNE5BJ26CV3QHFAMHANCNFSM4ISOTUHQ .

damienlevin commented 5 years ago

FYI, Apple players don't have support for the scheme mentioned in the links I provided (At least ATM. They do however support https://developer.apple.com/streaming/emsg-id3 ). I would suggest supporting both https://aomedia.org/emsg/ID3 and https://developer.apple.com/streaming/emsg-id3

See: https://forums.developer.apple.com/thread/118860

Here is a stream with an AOM scheme: https://storage.googleapis.com/dlevin/tmp/fmp4-emsg-id3/1567191409/master.m3u8

damienlevin commented 5 years ago

Hey @johnBartos, Any ETA for that feature ? Thanks

Damien

robwalch commented 4 years ago

@damienlevin Thanks for submitting the feature request.

Can you ask for AOM scheme support in Safari? 😉

This would be a great feature for v1.0.0, but I can't promise it will make the cut.

gogonowski commented 4 years ago

We originally submitted the request for ES (Elementary Stream ADTS) id3v2.4 metadata a few years ago. Now we move on to fMP4.

Here are some audio-only fMP4 URLs from the StreamS Live HLS Encoder, complete with compliant emsg id3v2.4 metadata. This has all been verified to be correct. They play and display in iTunes correctly, with the exception of the new xHE-AAC at the bottom of the list. Those require macOS 11 beta and Safari.

AAC-LC 320kbps fMP4 HLS Streams https://www.maxradio.ca/UHD/CKLGfMP4/320k/CKLG.m3u8 https://www.maxradio.ca/UHD/LG73fMP4/320k/LG73.m3u8 https://www.maxradio.ca/UHD/MaxfMP4/320k/Max.m3u8

HE-AACv1 80kbps fMP4 HLS Streams https://www.maxradio.ca/UHD/CKLGfMP4/80k/CKLG.m3u8 https://www.maxradio.ca/UHD/LG73fMP4/80k/LG73.m3u8 https://www.maxradio.ca/UHD/MaxfMP4/80k/Max.m3u8

These are new xHE-AAC streams, verified to play native in macOS 11 beta Safari. emsg id2v2.4 metadata is the same.

xHE-AAC 40kbps fMP4 HLS Streams https://www.maxradio.ca/UHD/CKLG/40k/CKLG.m3u8 https://www.maxradio.ca/UHD/LG73/40k/LG73.m3u8 https://www.maxradio.ca/UHD/MaxRadio/40k/Max.m3u8 https://www.maxradio.ca/UHD/NewWestRock/40k/NewWestRock.m3u8 https://www.maxradio.ca/UHD/UptownRadio/40k/Uptown.m3u8

We are planning on making a Metadata Timing Test available for fMP4 very soon. ES (ADTS Elementary Stream) Version is here: https://la2.indexcom.com/player/pad/hls/program.m3u8

Please let me know if you need any other information on this. Happy to help. /greg. StreamS

johndeu commented 3 years ago

Hey @johnBartos - did the emsg v1 support ever make it in? Just checking as we are in need of the 64 bit timestamp support now

gogonowski commented 3 years ago

We’ve been wondering the same thing.

Here is an audio-only HLS fMP4 stream with fully compliant emsg id3v2.4 metadata.

http://db2.indexcom.com/bucket/ram/00/01x/256k/program.m3u8

We also have HLS fMP4 xHE-AAC and ALAC/FLAC Lossless streams available for testing and development.

These streams play and display metadata and images correctly on macOS and the Music App.

I’ve been meaning to post these on hls.js.

There are many more available from various other providers, available upon request.

All the best.

/greg.

StreamS HiFi Radio

From: John Deutscher notifications@github.com Sent: Friday, 11 December, 2020 11:32 To: video-dev/hls.js hls.js@noreply.github.com Cc: gogonowski greg@indexcom.com; Comment comment@noreply.github.com Subject: Re: [video-dev/hls.js] Inband Timed Metadata for FMP4 (ID3 within Emsgv1) (#2360)

Hey @johnBartos https://github.com/johnBartos - did the emsg v1 support ever make it in? Just checking as we are in need of the 64 bit timestamp support now

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/video-dev/hls.js/issues/2360#issuecomment-743385344 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AC4LC5YZDJMCMJJ2GEYTJLTSUJXSLANCNFSM4ISOTUHQ . https://github.com/notifications/beacon/AC4LC5YTQEZRZR4FOSHW5TLSUJXSLA5CNFSM4ISOTUH2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOFRHSSAA.gif

robwalch commented 3 years ago

Hi @gogonowski and @johndeu, you can direct these questions to me. ID3 within Emsgv1 is not present in feature/v1.0.0 and will not make it before beta. It will most likely come in a minor following the release of v1.0.0, but that depends on whether we get contributions before then. I will update the issue milestone accordingly once we reach beta in the coming weeks. You can read more about the release here:

https://github.com/video-dev/hls.js/issues/2861 https://github.com/video-dev/hls.js/pull/3072

We do have a couple of PRs that add CMAF CC support. I believe that work is related and adds some of the tooling necessary to extract EMSG ID3, so it's not far off: https://github.com/video-dev/hls.js/pull/2623

robwalch commented 3 years ago

There are ways you can help! Please provide sample HLS streams containing Emsgv1 data; what the metadata contains and when each starts and ends. Confirming that the metadata is exposed in TextTracks by Safari when played in a video element with <video src='stream'/> would also help us get it right in hls.js. Thanks!

victor-at-work commented 3 years ago

I have a PR for this ready for review: https://github.com/victor-at-work/hls.js/pull/1, but it depends on some of the utilities added in #3016. I'll rebase when #3016 has been merged.

monyone commented 3 years ago

Hi @robwalch, Hasn't there been any progress on this issue?

I created fork for CMAF ID3 support without CMAF CC support from current master branch (v1.0.7). https://github.com/monyone/hls.js/tree/feature/cmaf-id3 (https://github.com/monyone/hls.js/commit/f361dd9476d7e5e89ffddc829b742fe3b1e94f24) It based on @victor-at-work's cmaf-id3 branch without CMAF CC support and rebase from current master branch.

May I create PR from this branch? Shulod I wait untill @victor-at-work's CMAF CC (v0.0.15) and CMAF ID3 (v1.1.0) has been merged?

victor-at-work commented 3 years ago

Hey @robwalch, I was planning on addressing @johndeu's comment on my PR, but if we're going to merge in @monyone's PR from v1 then I'll hold off from doing that, so please let me know :)

gogonowski commented 3 years ago

HLS fMP4 Audio-Only with emsg id3v2.4 Timed Metadata Reference Streams

There has been much question on how this is to be implemented. This is how you do it.

These streams have been produced with StreamS HLSdirect(tm) Live Encoder.

AAC-LC https://la2.indexcom.com/player/pad/fmp4/hls/128k/program.m3u8

AAC-HEv1 https://la2.indexcom.com/player/pad/fmp4/hls/64k/program.m3u8

xHE-AAC https://la2.indexcom.com/player/pad/fmp4/hls/40k/program.m3u8

FLAC - Lossless https://la2.indexcom.com/player/pad/fmp4/hls/FLAC/program.m3u8

This can be viewed in action here: https://la2.indexcom.com/player/eval https://la2.indexcom.com/player/pad/fmp4

/greg. StreamS

nicoweilelemental commented 2 years ago

Here is another live stream sample with ID3 tags over emsg v1 (in the video segments, not the audio ones), produced on AWS Elemental MediaPackage: https://d24rwxnt7vw9qb.cloudfront.net/out/v1/62a7ec8a0f3b4f19ad76eac54f2f2dce/cmaf-clear/index.m3u8