Try to add alpha channel support to the player by myself, but get an alpha buffer array which values are all zero from wasm

[LoS-Light]

I want to play a webm video(VP8) include alpha channel, so I try to add alpha channel support to the player by myself, then I dig into the source code.

~

1. First of all, I check the "vpx_image_t" struct definition from "vpx_image.h", so I can know that "planes[3]" and "stride[3]" are for alpha channel. ...

define VPX_PLANE_ALPHA 3 /*< A (Transparency) plane /

unsigned char *planes[4]; /*< pointer to the top left pixel for each plane / int stride[4]; /*< stride between rows for each plane /

~

2. Next, in "ogv-decoder-video-vpx.c", I modify some functions below static vpx_image_t copy_image(vpx_image_t src) { ... ... vpx_image_t dest = vpx_img_alloc(NULL, src->fmt, src->d_w, src->d_h, 16); copy_plane(dest, src, 0, width, height); copy_plane(dest, src, 1, chromaWidth, chromaHeight); copy_plane(dest, src, 2, chromaWidth, chromaHeight); copy_plane(dest, src, 3, width, height); //<---------- I add it return dest; } ... ... static int process_frame_return(void user_data) { ... ... ogvjs_callback_frame(image->planes[0], image->stride[0], image->planes[1], image->stride[1], image->planes[2], image->stride[2], image->planes[3], image->stride[3], //<---------- I add it image->w, height, chromaWidth, chromaHeight, image->d_w, image->d_h, // crop size 0, 0, // crop pos image->r_w, image->r_h); // render size ... ... }

~

3. Next, modify the declaration from "ogv-decoder-video.h" extern void ogvjs_callback_frame(unsigned char bufferY, int strideY, unsigned char bufferCb, int strideCb, unsigned char bufferCr, int strideCr, unsigned char bufferA, int strideA, //<---------- I add it int width, int height, int chromaWidth, int chromaHeight, int picWidth, int picHeight, int picX, int picY, int displayWidth, int displayHeight);

~

4. Next, go to the "ogv-decoder-video-callbacks.js" to modify the function "ogvjs_callback_frame" ogvjs_callback_frame: function(bufferY, strideY, bufferCb, strideCb, bufferCr, strideCr, bufferA, strideA, //<---------- I add it width, height, chromaWidth, chromaHeight, picWidth, picHeight, picX, picY, displayWidth, displayHeight) { ... ... var recycled = Module['recycledFrames'], frame, lenY = height strideY, lenCb = chromaHeight strideCb, lenCr = chromaHeight strideCr, lenA = height strideA; //<---------- I add it ... ... next['y']['bytes'].length === lenY && next['u']['bytes'].length === lenCb && next['v']['bytes'].length === lenCr && next['a']['bytes'].length === lenA //<---------- I add it ... ... 'v': { 'bytes': new Uint8Array(lenCr), 'stride': strideCr }, 'a': { //<---------- I add this part 'bytes': new Uint8Array(lenA), 'stride': strideA } ... ... copyAndTrim(frame['y']['bytes'], bufferY, strideY, height, picX, picY, picWidth, picHeight, 0); copyAndTrim(frame['u']['bytes'], bufferCb, strideCb, chromaHeight, chromaPicX, chromaPicY, chromaPicWidth, chromaPicHeight, 128); copyAndTrim(frame['v']['bytes'], bufferCr, strideCr, chromaHeight, chromaPicX, chromaPicY, chromaPicWidth, chromaPicHeight, 128); copyAndTrim(frame['a']['bytes'], bufferA, strideA, height, picX, picY, picWidth, picHeight, 0); //<---------- I add it

// And queue up the output buffer! Module['frameBuffer'] = frame;

console.log(frame); //<---------- I add it to log the frame }

~

5. After buiding, I play a test video(vp8 webm with alpha channel)(720x1280), and check the frame log from the chrome console, and in the 'a' property, Uint8Array length is 1024000 but the values are all zero. a: {bytes: Uint8Array(1024000), stride: 800}

~

I can't figure out where I am wrong, so I create this new issue to ask for help. Thank you very much.

[bvibber]

I don't know whether that technique works for getting at the alpha channel or not, but you should be aware that yuv-canvas does not support an alpha channel for the YUV to RGB conversion at this time.

You'd have to modify yuv-canvas to add this support.

[bvibber]

Closing in favor of older entry https://github.com/brion/ogv.js/issues/590