rockchipmpp: Add mppvpxalphadecodebin element

[julianbouzas]

Similar to gst-plugins-bad's codecalpha, this PR adds a new mppvpxalphadecodebin element that wraps the existing mppvideodec decoder to allow decoding the alpha channel of VP8 and VP9 videos.

In VP8 and VP9 videos, the alpha channel needs to be decoded separately and in parallel with the rest of the video, so 2 instances of mppvideodec are needed: one to decode the actualy video, and another one to decode the alpha channel. The way to do this in GStreamer is implement a bin that separates the alpha channel with the codecalhademux element, then decode separately alpha and video with mppvideodec, and finally combine them with the alphacombine element:

                                             .-> queue (alpha) -> mppvideodec -.
filesrc -> matroskademux -> codecalphademux -|                                 |-> alphacombine -> glimagesink
                                             `-> queue (video) -> mppvideodec -'

The video format after alphacombine can be either AV12 (NV12 with alpha) or A420 (I420 with alpha). If you use a sink that supports those formats such as glimagesink, you should be able to see the video with transparent background.

Here is a VP9 sample video with alpha:

You can test it with: https://www.dropbox.com/s/zsdhnum3rmlq3kg/file.webm?dl=0

gst-launch-1.0 filesrc location=file.webm ! matroskademux ! mppvpxalphadecodebin ! glimagesink

Or:

gst-launch-1.0 filesrc location=file.webm ! decodebin ! glimagesink

Here is the gstreamer dot graph of the first pipeline:

Without the mppvpxalphadecodebin element, it is not possible for the autoplug system to work with decodebin if the VP8/VP9 file has alpha channel (it will just decode the video ignoring the alpha channel). Having the mppvpxalphadecodebin element allows decodebin to decode alpha channel automatically, without needing to define the pipeline manually (requirement in some applications):

gst-launch-1.0 filesrc location=file.webm ! matroskademux ! codecalphademux name=ad ad.src ! queue ! mppvideodec ! co.sink ad.alpha ! queue ! mppvideodec ! co.alpha alphacombine name=co ! glimagesink

[JeffyCN]

Hi Julian,

thanks for the patch, looks good to me, but one more thing:

this plugin mainly targeting on gstreamer 1.14(buildroot 2018/debian 10), but the new element uses a few new APIs which are not available for 1.14(codecalphademux not in 1.14 too)

maybe we should add a minimal gst version for the new element in meson.build to rule it out from lower gstreamer versions?

[julianbouzas]

Hi Jeff,

Both codecalphademux and alphacombine elements are very new and they were added quite recently in gstreamer 1.19.

As you said, I can add a check in meson.build to avoid building the plugin if version is less than 1.19. However, since the mppvpxalphadecodebin element does not really use any new gstreamer API, I think the simplest solution would be to just add a version check before registering the element: if version is 1.19 or greater, then register the plugin, otherwise don't register it.

https://github.com/JeffyCN/rockchip_mirrors/commit/bba47cb4992bcf4571573352136b5818a7085d0e#diff-9ff859ddb0e764b7db13f27d3c2b058ebfc6055fdd680973dcc20ecc0245279eR387

Let me know if you are happy with that, otherwise I can add a meson check and avoid building the element.

[JeffyCN]

Hi Julian, mppvpxalphadecodebin is using a few new APIs, e.g. GST_ELEMENT_REGISTER_DECLARE, so it could not build with old gst versions(tested with 1.14)

other options would be converting those APIs to older versions or add them to https://github.com/JeffyCN/rockchip_mirrors/blob/gstreamer-rockchip/gst-libs/gst/gst-compat-private.h

[julianbouzas]

Hi Jeff,

I forgot to remove that line in the PR. GST_ELEMENT_REGISTER_DECLARE is used along with GST_ELEMENT_REGISTER_DEFINE to register an element (it is the new way to do it). However, it is not needed in the PR because I manually use gst_element_register like the rest of the elements.

I just updated the PR without that define, it should compile for 1.14 now.

[JeffyCN]

hi, still a few errors with 1.14: ../gst/rockchipmpp/gstmppalphadecodebin.c: In function ‘gst_mpp_alpha_decode_bin_constructed’: ../gst/rockchipmpp/gstmppalphadecodebin.c:159:3: error: implicit declaration of function ‘gst_clear_object’; did you mean ‘g_clear_object’? [-Werror=implicit-function-declaration] gst_clear_object (&sink_pad); ^~~~ g_clear_object ../gst/rockchipmpp/gstmppalphadecodebin.c: In function ‘gst_mpp_alpha_decode_bin_class_init’: ../gst/rockchipmpp/gstmppalphadecodebin.c:209:3: error: implicit declaration of function ‘gst_type_mark_as_plugin_api’; did you mean ‘g_type_plugin_use’? [-Werror=implicit-function-declaration] gst_type_mark_as_plugin_api (GST_TYPE_MPP_ALPHA_DECODE_BIN, 0); ^~~~~~~ g_type_plugin_use

and this plugin's gst require version is 1.10, so i'm not sure is there any more errors with older gst version... but since we(rockchip) mainly use it with gst 1.14 now, maybe i can increase the minimal gst version to 1.14 later.

[julianbouzas]

Hi,

The gst_clear_object(&object) API essentially just does this:

if (*object) {
  gst_object_unref (*object);
  *object = NULL;
}

So I updated the MR to use gst_object_unref with an if check, instead of directly using gst_clear_object.

Regarding gst_type_mark_as_plugin_api, it is just an API to mark the class as a plugin and make the documentation pretty. Since it is not used in the other plugins of this project, I removed it to make everything more consistent.

The MR should compile now for 1.14, and I think it should compile too for 1.10.

Let me know if you have more compiling issues.

[JeffyCN]

ok, thanks for the patch :)