Support packaging VP9 with alpha

[d8bauxit8]

I have a video (Codecs: AAC, Timecode, HEVC with Alpha) with transparent background (I continuously scroll a given text over the transparent background) which I want to play with DRM content protection.

System info

Operating System: MacOs Big Sur 2,6 GHz 6-Core Intel Core i7 Shaka Packager Version: v2.5.1-9f11077-release

Issue and steps to reproduce the problem

I converted this video (which I mentioned above) to VP9 with ffmpeg. The converted video is worked on my local environment (my webm video has still transparent background).

ffmpeg -i eme-test-alpha.mp4 \
-strict -2 -c:a opus \
-c:v libvpx-vp9 -profile:v 0 \
-vf colorkey=black -pix_fmt yuva420p -keyint_min 72 -g 72 \
-tile-columns 4 -frame-parallel 1 -speed 1 \
-auto-alt-ref 1 -lag-in-frames 25 \
-minrate 600k -maxrate 600k -bufsize 600k -b:v 600k \
-y vp9_600.webm

After that I used shaka-packager to create Widevine encryption with test environment (like here), but the transparent background is changed to black color.

Packager Command:

packager \
in=vp9_600.webm,stream=audio,output=audio.webm \
in=vp9_600.webm,stream=video,output=vp9_360p.webm \
--enable_widevine_encryption \
--key_server_url https://license.uat.widevine.com/cenc/getcontentkey/widevine_test \
--content_id 7465737420636f6e74656e74206964 \
--signer widevine_test \
--aes_signing_key 1ae8ccd0e7985cc0b6203a55855a1034afc252980e970ca90e5202689f947ab9 \
--aes_signing_iv d58ce954203b7c9a9a9d467f59839249 \
--protection_scheme cenc \
--mpd_output vp9.mpd

Extra steps to reproduce the problem? (1) I tried it with vp8 codec but it did not work either after shaka-packager using.

ffmpeg -i eme-test-alpha.mp4 \
-strict -2 -c:a opus \
-c:v libvpx -profile:v 0 \
-vf colorkey=black -pix_fmt yuva420p -keyint_min 72 -g 72 \
-tile-columns 4 -frame-parallel 1 -speed 1 \
-auto-alt-ref 0 -lag-in-frames 25 \
-minrate 600k -maxrate 600k -bufsize 600k -b:v 600k \
-y vp8_600.webm

Do you have any ideas or advice?

What is the expected result? I expected my background of video to be transparent.

What happens instead? My background of video turned black.

https://user-images.githubusercontent.com/25309438/215476016-51931b21-2b00-4fc0-8f04-b56ab1881eb5.mp4

[joeyparrish]

I seriously doubt that transparent videos are widely supported in decoders, and for encrypted video, different decoders and compositing methods may be used.

If there's evidence that Shaka Packager is removing your transparency, that would be a bug for us. But AFAICT, it could be removed during transcoding (before Packager) or ignored by the decoder (after Packager).

[kedanielwu]

I experience the same behavior, but with different codec and also without encryption.

I have a vp8 webm video with alpha channel and it plays correctly with Chrome natively, but after packaging the output webm file lost its alpha channel. You can check out with the links

original file: https://thumbplayer-76307.gzc.vod.tencent-cloud.com/testvideo/dash/alpha-webm-vp9/alpha.webm mpd: https://thumbplayer-76307.gzc.vod.tencent-cloud.com/testvideo/dash/alpha-webm-vp9/alpha-clear.mpd webm generated by Shaka: https://thumbplayer-76307.gzc.vod.tencent-cloud.com/testvideo/dash/alpha-webm-vp9/video-clear.webm

the original file in chrome://media correctly shows the alpha flag and file after packaging does not

I noticed in second case, vpx decoder of Chrome is failed to initialized without meaningful message, but the only different between two files are the later one went through Shaka

I use the following command to generate them

packager \
>   in=/media/alpha-webm-vp9/alpha.webm,stream=video,output=/media/alpha-webm-vp9/video-clear.webm \
>   --mpd_output /media/alpha-webm-vp9/alpha-clear.mpd

with latest Shaka running in docker

Edited: FFmpeg also reports the metadata for alpha mode is lost after packaging

if you have issue access those file, the original file is acquired from https://developer.chrome.com/blog/alpha-transparency-in-chrome-video/

[joeyparrish]

Okay, thanks. I've tagged the issue, but this is marked as a low priority for now. Feel free to contribute a PR if you'd like to get it fixed soon.

[aaronsvk]

any updates here?