Add support for alpha channel (transparency)

polybiusproxy / hxCodec

Haxe library for native video playback on HaxeFlixel and OpenFL.

https://lib.haxe.org/p/hxCodec

Mozilla Public License 2.0

109 stars 71 forks source link

Add support for alpha channel (transparency) #177

Open EliteMasterEric opened 2 years ago

EliteMasterEric commented 2 years ago

Certain video file formats (such as WEBM) have support for an alpha color channel. This results in the channels with lower alpha values being rendered with transparency. This would allow any sprites underneath the video to be visible.

See here for more information: https://developer.chrome.com/blog/alpha-transparency-in-chrome-video/

The use case for this is evident and powerful: encoding and playing a transparent video would allow for efficient and performant rendering of elaborate character animations not suited to spritesheets.

There are problems with this, however. The implementation of libvlc currently exports 0xFF for the alpha value regardless of settings. I am investigating alternatives.

trayfellow commented 2 years ago

hi mastereric

EliteMasterEric commented 1 year ago

Writing this down for posterity, but previous experimentation revealed that LibVLC 4.0 (currently an unstable version) has support for transparent decoding, but ONLY for the proprietary HAP codec. This has several downsides, notably poor compatibility with LibVLC 3.0 (currently in use) and very large file sizes (approximately 10x the size of an equivalent MP4 video).

For comparison, the other file formats which support transparency are:

GIF: Very high file sizes and poor color quality.
APNG: Doesn't play animation on LibVLC 3.

EliteMasterEric commented 1 year ago

Correction, HAP codec is compatible with LibVLC 3.0.

HAP creation script:

ffmpeg -framerate 24 -pattern_type sequence -i Week_7_cutscene_assets%04d.png -c:v hap -format hap_alpha output1.mov

nanjizal commented 1 year ago

Green screen is perfectly implementable in JS, by replacing a specific pixel color at run time. Jigsawx had webcam or video running in haxe flash reasonable well. There are certainly options using something similar to a double buffering to apply a mask ( perhaps composite with black and white image ), mask or greenscreen replace pixel options. Stuff like svg masks work well. In terms of old flash, we built a 3D website with papervision3D that had video people talking in 2D with transparent backgrounds within a 3D enviroment years ago. Therefore there must solutions even if the implementation is on the haxe not the hardware extern side. I am sure a shader could easily be used to achieve green screen or use a black and white video as a mask for a colour one.