Encrypt is not working

[iKalin]

Ben is possible to give me a short example for encrypting? I have all things installed but when I try to call function with -e option is noting happens! Thanks in advance! p.s. I put everything is needed to conf

[bentasker]

Hi Kalin,

Can you paste the command you're using? For example, if the script considers it's a live stream it won't encrypt (as the encryption happens after the fact)

Are you at least getting HLS video output? If so, in that directory is there a .key file?

Are you getting a key entry in the generated manifest?

[samueleastdev]

Hi Ben,

This script is awesome thanks, but I also cannot understand how encryption works could you please explain.

I am running.

./HLS-Stream-Creator.sh -i trailer.mov -s 10 -e -o trailer -b 272,872,1372,1872

I am not sure what option to pass the encryption param.

Would you be available for any freelance to get this working for me?

Thanks Sam

[bentasker]

Hi,

Are you getting any output (and or files) at all? If you're getting no video, could you direct stdout/stderr into a file and attach it?

Also, after running the command, what's the output of

 ls -l trailer

There's nothing to pass to the encryption param (I've made a note to update the README once we get to the bottom of this), you simply pass -e and the script will (or at least should) generate an encryption key, encrypt the segments with that and then update the manifest to tell the player to use that key.

That said, I've just run an encode with encryption enabled and encrypt hasn't run. Bear with me and I'll try and figure out why, but it'd be handy to know if your getting the same result as me (essentially the expected output but without any encryption, and no key generated)

[bentasker]

Actually, never mind - I think I've found it, just running some tests on the fix at the moment

[bentasker]

OK, so this took a bit longer than expected, the further down the rabbit hole the worse it got :)

If you do a pull now, you should find it works for you.

If you're interested in the gritty details they're in JIRA - http://projects.bentasker.co.uk/jira_projects/browse/HLS-20.html - but basically there were a few issues hidden away

Could you confirm whether encryption's now working for you?

[samueleastdev]

Hi Ben,

Thanks for looking into this for me, just ran some tests and it worked ;)

Here are two settings I am looking to add to the script the ability to generate thumbnails at certain segments with this command.

ffmpeg -i input.flv -vf fps=1/10 out%d.png

And then push the whole output to AWS with this command.

aws s3 sync . s3://s3bucket/trailer

I can pretty much run everything with this whole command but it would be nice to bind it all together in your script.

./HLS-Stream-Creator.sh -i intro2.mp4 -s 10 -e -o video -b 272,872,1372,1872 && ffmpeg -i intro2.mp4 -vf fps=1/10 video/out%d.png && aws s3 sync video s3://s3bucket/videotest

I seem to be having an issue with the encrypted video playback. I'm using video js so looking into this.

It plays back fine without encryption.

./HLS-Stream-Creator.sh -i intro2.mp4 -s 10 -o video -b 272,872,1372,1872 && ffmpeg -i intro2.mp4 -vf fps=1/10 video/out%d.png && aws s3 sync video s3://s3bucket/videotest

Thanks

Sam

[samueleastdev]

Hi Ben,

I am not 100 percent sure but does the .m3u8 file need to have the location of the key for example here is one of my m3u8 files from the AWS elastic transcoder.

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-ALLOW-CACHE:YES
#EXT-X-TARGETDURATION:13
#EXT-X-KEY:METHOD=AES-128,URI="720p.key",IV=0x8563b4d0d9588a4ea71a1ff24a2ebdf8
#EXTINF:12.046444,
720p00000.ts
#EXTINF:9.000000,
720p00001.ts
#EXTINF:9.000000,
720p00002.ts
#EXTINF:12.000000,
720p00003.ts
#EXTINF:9.000000,
720p00004.ts
#EXTINF:9.000000,
720p00005.ts
#EXTINF:12.000000,
720p00006.ts
#EXTINF:9.000000,
720p00007.ts
#EXTINF:9.000000,
720p00008.ts
#EXTINF:12.000000,
720p00009.ts
#EXTINF:9.000000,
720p00010.ts
#EXTINF:9.000000,
720p00011.ts
#EXTINF:12.000000,
720p00012.ts
#EXTINF:9.000000,
720p00013.ts
#EXTINF:9.000000,
720p00014.ts
#EXTINF:0.433333,
720p00015.ts
#EXT-X-ENDLIST

Thanks Sam

[samueleastdev]

Hi Ben,

Adding this line to all the main bitrate .m3u8 files solves the encryption playback issue ;)

#EXT-X-KEY:METHOD=AES-128,URI="video.key",IV=0x8563b4d0d9588a4ea71a1ff24a2ebdf8

Full file for one bitrate video_872.m3u8 looks like this.

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-ALLOW-CACHE:YES
#EXT-X-TARGETDURATION:12
#EXT-X-KEY:METHOD=AES-128,URI="video.key",IV=0x8563b4d0d9588a4ea71a1ff24a2ebdf8
#EXTINF:10.552222,
video_872_00000.ts
#EXTINF:10.677344,
video_872_00001.ts
#EXTINF:9.009011,
video_872_00002.ts
#EXTINF:10.719044,
video_872_00003.ts
#EXTINF:9.217544,
video_872_00004.ts
#EXTINF:10.176833,
video_872_00005.ts
#EXTINF:10.677333,
video_872_00006.ts
#EXTINF:11.302956,
video_872_00007.ts
#EXTINF:8.883878,
video_872_00008.ts
#EXTINF:9.134122,
video_872_00009.ts
#EXTINF:10.552211,
video_872_00010.ts
#EXTINF:10.135122,
video_872_00011.ts
#EXTINF:10.010000,
video_872_00012.ts
#EXTINF:10.468800,
video_872_00013.ts
#EXTINF:8.717044,
video_872_00014.ts
#EXT-X-ENDLIST

Thanks

Sam

[bentasker]

Hi Sam,

The manifest should have the key, yes - is it not being added? (it was on my tests).

Do you get anything relevant output to stderr at the end of the run? I had to make a change to the way the key was written into the manifest as the last set of commits, so it's possible I overlooked something.

It should be reasonably straight forward to add a flag so that you can specify a bucket and have the script push files there (assuming s3cmd is available etc). Or perhaps a flag so you can define a post-processing command to run (for example generating the thumbnails)

[bentasker]

Hmmm, actually, I think I've found an issue with the way the encryption works with ABR streams - http://projects.bentasker.co.uk/jira_projects/browse/HLS-22.html - will try and take a look at that later.

You should still be seeing the key tag written into the manifests thogh

Going to go over that function in depth to try and identify any other gremlins - http://projects.bentasker.co.uk/jira_projects/browse/HLS-21.html

[samueleastdev]

Ok cheers Ben ;)

[bentasker]

The latest commit should solve the IV issue (HLS-22)

I've only just noticed in your output, you manually added a static IV

#EXT-X-KEY:METHOD=AES-128,URI="video.key",IV=0x8563b4d0d9588a4ea71a1ff24a2ebdf8

Strictly speaking, playback should not work. The IV changes for each segment, so the IV should be incorrect (meaning decryption will fail). I found in testing this morning though, that VLC and ffplay seem to disregard the specified IV and simply use the segment number (or possibly try the specified IV and then fallback?)

I've not got to the bottom of why the key line isn't getting written into manifests for you though, they have been in all my tests. What's the exact commandline you're using (is it still the same as the one above)?

Assuming it is, after running the script, if you do

for manifest in video/*.m3u8
do
    echo "$manifest"
    # Insert the KEY at the 5'th line in the m3u8 file
    sed -i "5i #EXT-X-KEY:METHOD=AES-128,URI="video.key "$manifest"
done

What output do you get (and does the key line now appear in the manifests?)

[darkkillen]

I've got message "sed: 1: "./output/sample.m3u8": invalid command code ." and there is no key in manifest.

[bentasker]

What's the full commandline you're using to trigger?

[bentasker]

Actually, is that output of the block above?

What OS are you running this on? From the output it looks as though sed is interpreting the manifest name as part of the command code.

If you run

sed -i "5i #EXT-X-KEY:METHOD=AES-128,URI="video.key ./output/sample.m3u8

Do you get the same output?

[darkkillen]

I'm working on OSX. full command: "HLS-Stream-Creator.sh -i sample.mp4 -e -s 60 -p sample -t sample" output log: Generating Encryption Key Encrypting Segments usage: grep [-abcDEFGHhIiJLlmnOoqRSsUVvwxZ] [-A num] [-B num] [-C[num]] [-e pattern] [-f file] [--binary-files=value] [--color=when] [--context[=num]] [--directories=action] [--label] [--line-buffered] [--null] [pattern] [file ...] usage: grep [-abcDEFGHhIiJLlmnOoqRSsUVvwxZ] [-A num] [-B num] [-C[num]] [-e pattern] [-f file] [--binary-files=value] [--color=when] [--context[=num]] [--directories=action] [--label] [--line-buffered] [--null] [pattern] [file ...] usage: grep [-abcDEFGHhIiJLlmnOoqRSsUVvwxZ] [-A num] [-B num] [-C[num]] [-e pattern] [-f file] [--binary-files=value] [--color=when] [--context[=num]] [--directories=action] [--label] [--line-buffered] [--null] [pattern] [file ...] usage: grep [-abcDEFGHhIiJLlmnOoqRSsUVvwxZ] [-A num] [-B num] [-C[num]] [-e pattern] [-f file] [--binary-files=value] [--color=when] [--context[=num]] [--directories=action] [--label] [--line-buffered] [--null] [pattern] [file ...] Updating Manifests sed: 1: "./output/sample.m3u8": invalid command code .

The output folder contain: sample_00000.ts sample_00001.ts sample_00002.ts sample_00003.ts sample.key sample.m3u8

Manifest of sample.m3u8:

EXTM3U

EXT-X-VERSION:3

EXT-X-MEDIA-SEQUENCE:0

EXT-X-ALLOW-CACHE:YES

EXT-X-TARGETDURATION:62

EXTINF:61.680000,

sample_00000.ts

EXTINF:60.160000,

sample_00001.ts

EXTINF:58.520000,

sample_00002.ts

EXTINF:45.000000,

sample_00003.ts

EXT-X-ENDLIST

There are no "#EXT-X-KEY:METHOD=AES-128,URI=sample.key" if I add it my self. It's work normally.

[bentasker]

Ahhh, OK, that starts to make more sense.

The default tooling on OSX is different and doesn't support some of the features/flags in both GNU grep and sed.

Could you try installing GNU sed and grep to see if that resolves the issue for you

brew install gnu-sed --with-default-names
brew install grep --with-default-names

[darkkillen]

That's work like a charm. The problem solved. Thank you very much.

[bentasker]

Awesome, thanks for the update!

[bentasker]

As it looks like keys are now being written in properly, I'm going to close this off now. If that's not the case, please feel free to re-open