Cannot play SAMPLE-AES nor Segment AES fMP4 using VHS 2.2 non-native

RafalLukawiecki commented 4 years ago

Description

When playing back fMP4 HLS H264 stream that has both audio and video AES-Sample or AES-Segment encrypted using Apple mediafilesegmenter an uncaught exception is thrown in non-native playback using Chrome and Firefox. Safari plays back without any problems.

Steps to reproduce

Using hosted VHS 2.2.0 player at https://videojs-http-streaming.netlify.app/ try playing Sample encrypted stream at https://d2gw5qqm1tusud.cloudfront.net/pml-test-fmp4-sample-aes/m.m3u8
Try playing Segment encrypted https://d2gw5qqm1tusud.cloudfront.net/pml-test-fmp4-segment-aes/m.m3u8

Results

Expected

Normal playback.

Actual

Uncaught exception, and player hangs indefinitely.

Error output

Chrome:

Uncaught RangeError: byte length of Int32Array should be a multiple of 4
    at new Int32Array (<anonymous>)
    at new Decrypter (e078b76d-d433-48ec-a7ba-90c3f96bd0d8:548)
    at DecrypterWorker.self.onmessage (e078b76d-d433-48ec-a7ba-90c3f96bd0d8:647)
Decrypter @ e078b76d-d433-48ec-a7ba-90c3f96bd0d8:548
DecrypterWorker.self.onmessage @ e078b76d-d433-48ec-a7ba-90c3f96bd0d8:647

Firefox:

RangeError: attempting to construct out-of-bounds TypedArray on ArrayBuffer 95ef2b78-e7e8-ef4b-8ff6-a0491701a31e:548:29
    Decrypter blob:https://videojs-http-streaming.netlify.app/95ef2b78-e7e8-ef4b-8ff6-a0491701a31e:548
    onmessage blob:https://videojs-http-streaming.netlify.app/95ef2b78-e7e8-ef4b-8ff6-a0491701a31e:647

Additional Information

versions

videojs

7.8.3

browsers

Chrome 86, Firefox 81

OSes

macOS 10.15.7

plugins

vhs 2.2.0

gkatsev commented 4 years ago

Unfortunately, SAMPLE-AES isn't supported right now.

RafalLukawiecki commented 4 years ago

Thank you. Let me retest with segment AES.

RafalLukawiecki commented 4 years ago

I am afraid it won't play segment AES either. Please have a look at this stream: https://d2gw5qqm1tusud.cloudfront.net/pml-test-fmp4-segment-aes/m.m3u8 using your hosted player.

In Chrome the error is:

Uncaught TypeError: Cannot read property 'video' of undefined
    at handleSegmentBytes (videojs-http-streaming.js:9497)
    at Worker.decryptionHandler (videojs-http-streaming.js:9650)
handleSegmentBytes @ videojs-http-streaming.js:9497
decryptionHandler @ videojs-http-streaming.js:9650

In Firefox the error is:

Uncaught TypeError: tracks is undefined

Is this a problem with my stream or VHS/video.js? I am happy to provide more info if that helps.

PS. It plays in Safari without any issues.

gkatsev commented 4 years ago

AES-128 should work. We test it on a regular basis https://videojs-http-streaming.netlify.app/?debug=false&autoplay=false&muted=false&minified=false&liveui=false&partial=false&url=https%3A%2F%2Fplayertest.longtailvideo.com%2Fadaptive%2Foceans_aes%2Foceans_aes.m3u8&type=application%2Fx-mpegurl&keysystems=&buffer-water=false However, it's definitely possible that there's some kind of bug there.

RafalLukawiecki commented 4 years ago

That does not look like fragmented MP4 but more like MPEG-TS. Are you testing AES with fMP4? Please have a look at the stream I shared (https://d2gw5qqm1tusud.cloudfront.net/pml-test-fmp4-segment-aes/m.m3u8) to see if it plays for you. Do you think the issue is with my stream or VHS/VJS?

If it would be helpful, I can encode a few of those problematic fMP4 test streams so that you can see the issues on an ongoing basis.

RafalLukawiecki commented 4 years ago

I have retested just now using video.js 7.10.1 and the built-in VHS 2.2.3 and I am afraid Segment-AES still does not work for fMP4 containers.

stale[bot] commented 3 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

RafalLukawiecki commented 3 years ago

Still an issue with videojs 7.11.0 and vhs 2.4.1:

Cannot play SEGMENT-AES: https://d2gw5qqm1tusud.cloudfront.net/pml-test-fmp4-segment-aes/m.m3u8
Cannot play SAMPLE-AES: https://d2gw5qqm1tusud.cloudfront.net/pml-test-fmp4-sample-aes/m.m3u8

video-archivist-bot commented 3 years ago

Hey! We've detected some video files in a comment on this issue. If you'd like to permanently archive these videos and tie them to this project, a maintainer of the project can reply to this issue with the following commands:

for https://d2gw5qqm1tusud.cloudfront.net/pml-test-fmp4-segment-aes/m.m3u8: say @video-archivist-bot save zEpzwd
for https://d2gw5qqm1tusud.cloudfront.net/pml-test-fmp4-sample-aes/m.m3u8: say @video-archivist-bot save 9E3gE0

gkatsev commented 3 years ago

Unfortunately, we don't support SAMPLE-AES at all right now, and it does seem like we never got AES-128 working with fmp4 segments rather than TS segments.

freddyamsterdam commented 3 years ago

@RafalLukawiecki we were facing the same issue. After some trial and error, I figured out that the initiation vector seems to be the issue. I simply encoded our files with an empty string as an initiation vector and made sure to pass an empty IV to the playlist file:

#EXT-X-KEY:METHOD=AES-128,URI="pb-alpha-9cdffa16bbd5ccf0-0.key",IV=

Hope it helps.

Freddy

RafalLukawiecki commented 3 years ago

Interesting, @freddyamsterdam. I will try encoding a new video with an empty IV to see if that helps with our streams. In any case, if that works it would suggest there is a problem with how the IV value is read or used with fMP4 segments in video.js, which could be a good debugging hint. Thanks.

freddyamsterdam commented 3 years ago

@RafalLukawiecki it doesn't seem to work with multiple representation playlists. 😞 Will post an update if I manage to make any progress.

gkatsev commented 3 years ago

Oh, interesting that it works with an empty IV. It definitely works with ts segments, I wonder what's there that's failing for fmp4 segments.

freddyamsterdam commented 3 years ago

@gkatsev didn't mention before, but we're using TS segments 😢

gkatsev commented 3 years ago

@freddyamsterdam TS segments should work with IV properly. Did you find it didn't? If so, can you open a separate issue? And if you're able to provide a sample stream, that would be awesome.

RafalLukawiecki commented 3 years ago

Thank you, both, for following up. I am unsure if and what I can do to help troubleshoot it further. The test stream is still available.

freddyamsterdam commented 3 years ago

@gkatsev sure thing!

@RafalLukawiecki did the empty IV not work out for you?

RafalLukawiecki commented 3 years ago

So sorry, I have not rendered another test stream with an empty IV, yet. I will do that.

iroha7941 commented 3 years ago

The issue with fMP4 appears to be that the init segments are not being decrypted in http-streaming. This also explains why TS works but fMP4 does not, since init segments are only used for fMP4 but not for TS.

The example2.zip reproducer in https://github.com/videojs/video.js/issues/7051 shows that encrypting the init segment works with the Apple native decoder. Also the HLS spec (RFC 8216) notes that the init segments are encrypted: https://tools.ietf.org/html/draft-pantos-hls-rfc8216bis-08#section-4.4.4.4

I started writing a patch but I suspect someone more familiar with the project might be able to fix this faster/better.

RafalLukawiecki commented 3 years ago

@iroha7941, unless someone else comes up with a solution, I would be happy to help test your patch.

gkatsev commented 3 years ago

@iroha7941 oh, that makes a lot of sense. We probably didn't think that the init segments could also be encrypted when we added fmp4 support, and we didn't have any test content back then. If you're able to provide a patch that would be amazing! We'll happily review and help out, feel free to ask questions in the #playback channel on our slack: http://slack.videojs.com

iroha7941 commented 3 years ago

I created patches for http-streaming and m3u8-parser that enable video.js to playback the encrypted fMP4 files I've created. I created a repo containing a patched build of video.js and links to the branches in my forked repos which contain the patches: https://github.com/iroha7941/video.js-build

I have not yet done a lot of testing (and I haven't really tested for byte-range segments), and am still having problems running unit tests for http-streaming, but this seems to work for me.

gkatsev commented 3 years ago

@iroha7941 I finally got around to looking at the m3u8-parser PR and didn't see it working with my local VHS copy, then I noticed that you had VHS related changes as well. Is there a reason why you didn't make a PR for it? Is it because it's multiple commits behind main? I can confirm that your changes do work.

iroha7941 commented 3 years ago

@gkatsev I hadn't created a PR for VHS as I still haven't been able to run testsuite successfully (many tests fail with "vhs is undefined"), let alone add appropriate tests. I'm guessing this is just a problem with my setup or something (I'm still new to Javascript/npm development), but I haven't had time to look into it more. When I have a patch with appropriate tests ready I'll sumbit a PR. Sorry for the delay.

gkatsev commented 3 years ago

If you make a PR we can try and help figure out what's wrong. Either way, thanks!

index23 commented 2 years ago

Hi @gkatsev, did you get to play SAMPLE-AES encrypted fmp4 files with HLS?

ghost commented 2 years ago

@iroha7941 I have the same question. I encrypt every of my ABR tracks with a different SAMPLE-AES key and pass it to the m3u8 playlist of that specific track. Sadly, I also don't get VideoJS to play the Stream :( Would be awesome to finally see this feature

videojs / video.js