video-dev / hls.js

HLS.js is a JavaScript library that plays HLS in browsers with support for MSE.
https://hlsjs.video-dev.org/demo
Other
14.82k stars 2.57k forks source link

I don't understand the Key retrieval process for AES-128 (E-HLS) #6117

Closed ghost closed 9 months ago

ghost commented 9 months ago

What do you want to do with Hls.js?

Hello,

I would like to be able to intercept the key retrieval process.

Previously in videojs 7.21.5 I did this:

videojs.Vhs.xhr.beforeRequest = (options) => {

      if (options.uri && options.uri.includes('kms://')) {
        options.headers = { Authorization: this.file_response.keymerchant_token };
        options.withCredentials = true;

        options.uri = options.uri.replace('kms://', 'https://kms.mydomain.com/');
      }
      else {
        options.headers = { Authorization: this.file_response.access_token };
        options.withCredentials = true;
      }

      return options;
    };

Now since I switched over to hls.js I dropped the function above in favor of this:

hlsjsConfig: {
          debug: true,
          enableWorker: true,
          lowLatencyMode: false,
          backBufferLength: 90,
          xhrSetup: (xhr: any, url: string) => {
            if (url.includes('kms://')) {
              xhr.open('GET', url.replace('kms://', 'https://kms.mydomain.com/'), true);
              xhr.setRequestHeader('Authorization', this.file_response.keymerchant_token);
            } else {
              xhr.open('GET', url, true);
              xhr.setRequestHeader('Authorization', this.file_response.access_token);
            }
            xhr.withCredentials = true;
          },
        },

I can see that the Key is getting received at my browser tab and that the content is fine, but hls.js returns the following error:

HLS.js error: otherError - fatal: true - internalException

So what's the problem here? I basically have the following at my m3u8 playlists:

#EXTM3U
#EXT-X-VERSION:7
#EXT-X-TARGETDURATION:4
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-PLAYLIST-TYPE:VOD
#EXT-X-INDEPENDENT-SEGMENTS
#EXT-X-MAP:URI="Hen90dSsTr1KYP9JtzhPX4gmHDzZcZOSthQTYU2tfoelZo72GA8Evo6F5PlGV1Yz"
#EXT-X-KEY:METHOD=AES-128,URI="kms://c6707f70-e143-4516-941a-9f39c7503454",IV=0e3622be04f7ca9ce0f20432c9ebb5f1
#EXTINF:4.000000,
4TiTbAxOpemu5GnFuPo8q87AzzMwV3Urw8Spbe3CFuB2mcpBPFGGS1ReUwT8vQoU
#EXTINF:4.000000,
IIMHtQZ3B1oqDUlciEjFmVBypMDtZpndWwH5UUFDTpinGDzCzwiQKFG6bn4AisLA
#EXTINF:4.000000,
tHYGxtC45e4ef8jeKVllOepvZnAX7GOGs5lyQG25PJipKVayIXb18zI7bUBObIml

The reason why I have it setup using "kms://" is that it simply should act as a placeholder. In case I want to change my Domain or so. Hard-coding the URL into the playlist is kinda ugly in my opinion.

Can somebody tell me why I don't have any issue with that setup using VideoJS, but hls.js isn't working this way? I mean the xhr replace step, is basically the same, and I also get back the key from my backend as a 16 bit binary representation of the key.

This is what I get at my debug log:

[log] > Debug logs enabled for "Hls instance" in hls.js version 1.5.1 hls.mjs:414:21
[log] > attachMedia hls.mjs:27917:11
[log] > [buffer-controller] created media source: MediaSource hls.mjs:17801:11
[log] > stopLoad hls.mjs:27975:11
[log] > loadSource:https://some.cdn.com/hls/master_avc1.640029_VU8xX9kcCr5D3SDw24pDVYYF85LQm2CGYdxZXhyskxdU88oNgkvdt2NL1InA9xqH.m3u8 hls.mjs:27945:11
[log] > [stream-controller]: Trigger BUFFER_RESET hls.mjs:26957:9
[log] > [buffer-controller] Media source opened hls.mjs:17681:11
[log] > [level-controller]: manifest loaded, 9 level(s) found, first bitrate: 6842572 hls.mjs:25633:13
[log] > setting initial bwe to 5000000 hls.mjs:6948:13
[log] > [buffer-controller] 2 bufferCodec event(s) expected hls.mjs:17793:9
[log] > Setting autoLevelCapping to 8: 1080p@6842572 for media 1545x411 hls.mjs:21679:17
[log] > set autoLevelCapping:8 hls.mjs:28221:13
[log] > startLoad(-1) hls.mjs:27964:11
[log] > [abr] picked start tier {"codecSet":"avc1,mp4a","videoRanges":["SDR"],"preferHDR":false,"minFramerate":24,"minBitrate":442230} hls.mjs:7333:13
[info] > [abr] switch candidate:8->4 adjustedbw(5000000)-bitrate=3703274 ttfb:0.1 avgDuration:0.0 maxFetchDuration:4.0 fetchDuration:0.1 firstSelection:true codecSet:avc1,mp4a videoRange:SDR hls.loadLevel:-1 hls.mjs:7410:17
[log] > [level-controller]: Switching to level 4 (720p SDR avc1,mp4a @1296726) from level -1 hls.mjs:25709:9
[log] > [audio-track-controller]: Updating audio tracks, 2 track(s) found in group(s): a-3 hls.mjs:16528:11
[log] > [audio-track-controller]: Switching to audio-track 0 "Deutsch (5.1)" lang:de group:a-3 channels:2 hls.mjs:16634:9
[log] > [audio-stream-controller]: Reset loading state hls.mjs:10196:9
[log] > [audio-stream-controller]: STOPPED->IDLE hls.mjs:10306:11
[log] > [audio-stream-controller]: IDLE->WAITING_TRACK hls.mjs:10306:11
[log] > [subtitle-track-controller]: Updating subtitle tracks, 4 track(s) found in "s-0" group-id hls.mjs:17337:11
[log] > [level-controller]: Loading level index 4 with https://some.cdn.com/hls/X3VrB0WqvXeS2iPd7e5jsxiqDNd0BdaqXEg6a3qnNq7pNP3QHNHHN74RWJiQZ62Q hls.mjs:25844:11
[log] > [stream-controller]: STOPPED->IDLE hls.mjs:10306:11
[log] > [audio-track-controller]: loading audio-track playlist 0 "Deutsch (5.1)" lang:de group:a-3 hls.mjs:16708:11
[log] > [audio-stream-controller]: WAITING_TRACK->STOPPED hls.mjs:10306:11
[log] > [audio-stream-controller]: STOPPED->WAITING_TRACK hls.mjs:10306:11
[log] > [subtitle-stream-controller]: STOPPED->IDLE hls.mjs:10306:11
[log] > [stream-controller]: Level 4 loaded [0,433][part-433--1], cc [0, 0] duration:1733.708333 hls.mjs:27007:9
[log] > [buffer-controller] Updating Media Source duration to 1733.708 hls.mjs:18350:11
[log] > [stream-controller]: Loading fragment initSegment cc: 0 of [0-433] level: 4, target: 0 hls.mjs:9546:9
[log] > [stream-controller]: IDLE->FRAG_LOADING hls.mjs:10306:11
[log] > [audio-track-controller]: Audio track 0 "Deutsch (5.1)" lang:de group:a-3 loaded [0-433] hls.mjs:16467:9
[log] > [audio-stream-controller]: Audio track 0 loaded [0,433][part-433--1],duration:1733.834667 hls.mjs:16016:9
[log] > [audio-stream-controller]: WAITING_TRACK->IDLE hls.mjs:10306:11
[log] > [audio-stream-controller]: Loading fragment initSegment cc: 0 of [0-433] track: 0, target: 0 hls.mjs:9546:9
[log] > [audio-stream-controller]: IDLE->FRAG_LOADING hls.mjs:10306:11
[log] > [stream-controller]: FRAG_LOADING->IDLE hls.mjs:10306:11
[log] > [stream-controller]: Loading key for 0 of [0-433], level 4 hls.mjs:9486:11
[log] > [stream-controller]: IDLE->KEY_LOADING hls.mjs:10306:11
[log] > [stream-controller]: Loading fragment 0 cc: 0 of [0-433] level: 4, target: 0 hls.mjs:9546:9
[log] > [stream-controller]: KEY_LOADING->FRAG_LOADING hls.mjs:10306:11
[log] > [audio-stream-controller]: FRAG_LOADING->IDLE hls.mjs:10306:11
[log] > [audio-stream-controller]: Loading key for 0 of [0-433], track 0 hls.mjs:9486:11
[log] > [audio-stream-controller]: IDLE->KEY_LOADING hls.mjs:10306:11
[log] > [audio-stream-controller]: Loading fragment 0 cc: 0 of [0-433] track: 0, target: 0 hls.mjs:9546:9
[log] > [audio-stream-controller]: KEY_LOADING->FRAG_LOADING hls.mjs:10306:11
[log] > [transmuxer-interface, main]: Starting new transmux session for sn: 0 p: -1 level: 4 id: 1
        discontinuity: true
        trackSwitch: true
        contiguous: false
        accurateTimeOffset: true
        timeOffset: 0
        initSegmentChange: true hls.mjs:15448:13
[log] > [decrypter]: JS AES decrypt hls.mjs:9021:11
[log] > [stream-controller]: FRAG_LOADING->ERROR hls.mjs:10306:11
[log] > [audio-stream-controller]: FRAG_LOADING->ERROR hls.mjs:10306:11
[log] > stopLoad hls.mjs:27975:11

At my Network tab I don't see any loading issue, not for the decryption key and also not for the segments, everything gets answered with either http200 (GET) or http204 (OPTIONS)

Thanks in advance

ghost commented 9 months ago

After some more digging I can confirm that HLS with AES-128 encryption appears to be broken, even if I directly provide the key as a file the hls.js player will fall into fatal error state. Can please somebody validate this.

robwalch commented 9 months ago

The demo page AES-128 assets are playing back fine without any exceptions.

What is the internalException you are experiencing (what code is throwing, where)?

The stack trace should be printed out in the dev console. You can find the exception by enabling "pause on caught exceptions" in the debugger. Or, disable debugging for it to not be caught (assuming it's thrown in an event callback or your xhrSetup callback).

Can you share a page or codepen that reproduces the issue?

ghost commented 9 months ago

@robwalch, that's good to hear ^^ Well, I already pasted the debug output, or what did you mean? Sadly, this is all I've got. If I skip through the exception using "Pause on caught exceptions" in my Chrome browser, it stops at several places, but I'm not sure where to look at. Putting this at CodePen isn't possible due to several reasons, especially because of using licensed software components. But I can share my video-player-component.ts setup, please see this paste:

https://pastebin.com/EARaDLcz

Thanks in advance

ghost commented 9 months ago

I now checked my stream against the demo page of hls.js -> https://hlsjs.video-dev.org/demo

But also here the stream won't play, despite that I disabled all JWT authentication, the segments getting pulled, the key also but no playback, it basically stops with the following error:

Error event: 
Object { type: "otherError", details: "internalException", err: RangeError, error: RangeError, fatal: true }
​
details: "internalException"
​
err: RangeError: offset is outside the bounds of the DataView
​
error: RangeError: offset is outside the bounds of the DataView
​​
columnNumber: 28
​​
fileName: "https://hlsjs.video-dev.org/dist/hls.js"
​​
lineNumber: 9019
​​
message: "offset is outside the bounds of the DataView"
​​
stack: "uint8ArrayToUint32Array_@https://hlsjs.video-dev.org/dist/hls.js:9019:28\ndecrypt@https://hlsjs.video-dev.org/dist/hls.js:9163:29\nsoftwareDecrypt@https://hlsjs.video-dev.org/dist/hls.js:9340:46\npush@https://hlsjs.video-dev.org/dist/hls.js:15065:41\npush@https://hlsjs.video-dev.org/dist/hls.js:16035:42\n_handleFragmentLoadProgress@https://hlsjs.video-dev.org/dist/hls.js:27717:18\nprogressCallback@https://hlsjs.video-dev.org/dist/hls.js:9641:16\n_doFragLoad/result<@https://hlsjs.video-dev.org/dist/hls.js:9948:29\npromise callback*_doFragLoad@https://hlsjs.video-dev.org/dist/hls.js:9945:130\n_loadFragForPlayback@https://hlsjs.video-dev.org/dist/hls.js:9643:12\nloadFragment@https://hlsjs.video-dev.org/dist/hls.js:9630:12\nloadFragment@https://hlsjs.video-dev.org/dist/hls.js:27444:56\ndoTickIdle@https://hlsjs.video-dev.org/dist/hls.js:27430:12\ndoTick@https://hlsjs.video-dev.org/dist/hls.js:27320:14\ntick@https://hlsjs.video-dev.org/dist/hls.js:7870:14\nsetInterval handler*setInterval@https://hlsjs.video-dev.org/dist/hls.js:7832:35\nstartLoad@https://hlsjs.video-dev.org/dist/hls.js:27247:14\nstartLoad/<@https://hlsjs.video-dev.org/dist/hls.js:28577:20\nstartLoad@https://hlsjs.video-dev.org/dist/hls.js:28576:31\nfilterAndSortMediaOptions@https://hlsjs.video-dev.org/dist/hls.js:26308:18\nonManifestLoaded@https://hlsjs.video-dev.org/dist/hls.js:26168:12\nemit@https://hlsjs.video-dev.org/dist/hls.js:15542:36\nemit@https://hlsjs.video-dev.org/dist/hls.js:28464:28\ntrigger@https://hlsjs.video-dev.org/dist/hls.js:28468:21\nhandleMasterPlaylist@https://hlsjs.video-dev.org/dist/hls.js:4264:11\nonSuccess@https://hlsjs.video-dev.org/dist/hls.js:4202:19\nreadystatechange@https://hlsjs.video-dev.org/dist/hls.js:25267:28\nEventHandlerNonNull*openAndSendXhr@https://hlsjs.video-dev.org/dist/hls.js:25204:54\nloadInternal@https://hlsjs.video-dev.org/dist/hls.js:25185:14\nload@https://hlsjs.video-dev.org/dist/hls.js:25151:12\nload@https://hlsjs.video-dev.org/dist/hls.js:4215:14\nonManifestLoading@https://hlsjs.video-dev.org/dist/hls.js:4068:12\nemit@https://hlsjs.video-dev.org/dist/hls.js:15542:36\nemit@https://hlsjs.video-dev.org/dist/hls.js:28464:28\ntrigger@https://hlsjs.video-dev.org/dist/hls.js:28468:21\nloadSource@https://hlsjs.video-dev.org/dist/hls.js:28558:12\nloadSelectedStream@https://hlsjs.video-dev.org/dist/hls-demo.js:24456:8\napplyConfigEditorValue@https://hlsjs.video-dev.org/dist/hls-demo.js:25492:4\nonclick@https://hlsjs.video-dev.org/demo/?src=https%3A%2F%2Fpath_to_my_m3u8_file&demoConfig=eyJlbmFibGVTdHJlYW1pbmciOnRydWUsImF1dG9SZWNvdmVyRXJyb3IiOnRydWUsInN0b3BPblN0YWxsIjp0cnVlLCJkdW1wZk1QNCI6ZmFsc2UsImxldmVsQ2FwcGluZyI6LTEsImxpbWl0TWV0cmljcyI6LTF9:1:1\n"
​​
<prototype>: RangeError.prototype { stack: "", … }
​​​
constructor: function RangeError()
​​​​
length: 1
​​​​
name: "RangeError"
​​​​
prototype: RangeError.prototype { stack: "", … }
​​​​​
constructor: function RangeError()
​​​​​
message: ""
​​​​​
name: "RangeError"
​​​​​
stack: ""
​​​​​
<prototype>: Error.prototype { stack: "", … }
​​​​
<prototype>: function Error()
​​​
message: ""
​​​
name: "RangeError"
​​​
stack: ""
​​​
<prototype>: Error.prototype { stack: "", … }
​
fatal: true
​
type: "otherError"
​
<prototype>: Object { … }
main.js:734:12

This is how I encrypt my segments using python:

                        exec_command(
                            f'openssl aes-128-cbc '
                            f'-e -in "{old_full_segment_path}" -out "{new_full_segment_path}" '
                            f'-nosalt -iv {enc_iv} -K {enc_key}'
                        )

I do this for all segments except the init-segment, this will stay unencrypted

ghost commented 9 months ago

I also noticed the following at the hls.js demo page, if I check out the m3u8 playlists here:

  1. https://playertest.longtailvideo.com/adaptive/customIV/prog_index.m3u8 -> has IV
  2. https://playertest.longtailvideo.com/adaptive/oceans_aes/oceans_aes-audio=65000-video=370000.m3u8 -> does not have IV
  3. https://playertest.longtailvideo.com/adaptive/oceans_aes/oceans_aes-audio=65000-video=2042000.m3u8 -> does not have IV

Why do some of these streams have an Initialization vector and others don't ???

Tried with the test: "AES-128 encrypted, ABR"

Maybe I should also notice that I only use fmp4 (.m4s segments), not .ts segments.

robwalch commented 9 months ago

Have you tried setting enableSoftwareAES to false?

robwalch commented 9 months ago

Why do some of these streams have an Initialization vector and others don't ???

Those fall back to using the segments' media sequence number as the IV. (It's an older convention of the HLS spec.)

ghost commented 9 months ago

@robwalch Yes, I also tried that, but I'm currently onto something:

Take a look at this:

https://hlsjs.video-dev.org/demo/?src=https%3A%2F%2Fd12zt1n3pd4xhr.cloudfront.net%2Fdev%2Faes128m4s-en.m3u8&demoConfig=eyJlbmFibGVTdHJlYW1pbmciOnRydWUsImF1dG9SZWNvdmVyRXJyb3IiOnRydWUsInN0b3BPblN0YWxsIjpmYWxzZSwiZHVtcGZNUDQiOmZhbHNlLCJsZXZlbENhcHBpbmciOi0xLCJsaW1pdE1ldHJpY3MiOi0xfQ==

the stream is not mine, but I was able to detect a difference at the EXT-X-KEY line of the playlist, as you can see here:

https://d12zt1n3pd4xhr.cloudfront.net/dev/aes128m4s/360p.m3u8

Could it be that hlsjs expects the IV to be a hex string ? because in my playlist its not a hex string, but this is fixable, what do you think ?

What's also kinda funny, is that fact when switching these two lines:

EXT-X-MAP:URI="Hen90dSsTr1KYP9JtzhPX4gmHDzZcZOSthQTYU2tfoelZo72GA8Evo6F5PlGV1Yz"

EXT-X-KEY:METHOD=AES-128,URI="kms://c6707f70-e143-4516-941a-9f39c7503454",IV=0e3622be04f7ca9ce0f20432c9ebb5f1

over to :

EXT-X-KEY:METHOD=AES-128,URI="kms://c6707f70-e143-4516-941a-9f39c7503454",IV=0e3622be04f7ca9ce0f20432c9ebb5f1

EXT-X-MAP:URI="Hen90dSsTr1KYP9JtzhPX4gmHDzZcZOSthQTYU2tfoelZo72GA8Evo6F5PlGV1Yz"

If I first reference the key, I get a segment decryption error, but not :

HLS.js error: otherError - fatal: true - internalException

as before ...

robwalch commented 9 months ago

Could it be that hlsjs expects the IV to be a hex string ?

Yes.

In m3u8-parser:

const decryptiv = keyAttrs.hexadecimalInteger('IV');

From the HLS spec:

IV

The value is a hexadecimal-sequence that specifies a 128-bit
unsigned integer Initialization Vector to be used with the key.

because in my playlist its not a hex string, bit this is fix able

What is it?

ghost commented 9 months ago

Huraaaaaaaaaaa! Exactly that was the issue :D ... hls.js expect the IV to be in this format:

0x593b93926b1085cab158c02f69e5c9db

Woow, that was a ride of a bug search now xD

@robwalch Thanks for your interest in my problem ❤️ !!!

*previously it was a hex string but without the 0x indicator ...

ghost commented 9 months ago

Closed, as issue is resolved for me