A23 support - Githubissues

Hi, is it possible to run this on A23? I applied this patch to kernel:

diff --git a/arch/arm/boot/dts/sun8i-a23-a33.dtsi b/arch/arm/boot/dts/sun8i-a23-a33.dtsi
index 4461d5098..2fadc02d5 100644
--- a/arch/arm/boot/dts/sun8i-a23-a33.dtsi
+++ b/arch/arm/boot/dts/sun8i-a23-a33.dtsi
@@ -152,6 +152,17 @@ ve_sram: sram-section@0 {
                        };
                };

+                video-codec@1c0e000 {
+                        compatible = "allwinner,sun8i-a33-video-engine";
+                        reg = <0x01c0e000 0x1000>;
+                        clocks = <&ccu CLK_BUS_VE>, <&ccu CLK_VE>,
+                                 <&ccu CLK_DRAM_VE>;
+                        clock-names = "ahb", "mod", "ram";
+                        resets = <&ccu RST_BUS_VE>;
+                        interrupts = <GIC_SPI 58 IRQ_TYPE_LEVEL_HIGH>;
+                        allwinner,sram = <&ve_sram 1>;
+                };
+
                dma: dma-controller@1c02000 {
                        compatible = "allwinner,sun8i-a23-dma";
                        reg = <0x01c02000 0x1000>;

and... it's not working! Timeout when waiting for media request!

VLC media player 3.0.16 Vetinari (revision 3.0.13-8-g41878ff4f2)
[00575a48] vlcpulse audio output error: PulseAudio server connection failure: Connection refused
[004f9b98] main libvlc: Running vlc with the default interface. Use 'cvlc' to use vlc without interface.
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-user'
libva info: VA-API version 1.10.0
libva info: User environment variable requested driver 'v4l2_request'
libva info: Trying to open /usr/lib/arm-linux-gnueabihf/dri/v4l2_request_drv_video.so
libva info: Found init function __vaDriverInit_1_10
libva info: va_openDriver() returns 0
[007b2ab8] avcodec decoder: Using v4l2-request for hardware decoding
v4l2-request: Timeout when waiting for media request
[h264 @ 0x7d8300] Failed to end picture decode issue: 1 (operation failed).
[b2b05650] main filter error: Failed to create video converter
[h264 @ 0x7d8300] hardware accelerator failed to decode picture
v4l2-request: Timeout when waiting for media request
[h264 @ 0x81e8a0] Failed to end picture decode issue: 1 (operation failed).
[h264 @ 0x81e8a0] hardware accelerator failed to decode picture
v4l2-request: Timeout when waiting for media request
[h264 @ 0x860380] Failed to end picture decode issue: 1 (operation failed).
[h264 @ 0x860380] hardware accelerator failed to decode picture
v4l2-request: Timeout when waiting for media request
[h264 @ 0x7d8300] Failed to end picture decode issue: 1 (operation failed).
[h264 @ 0x7d8300] hardware accelerator failed to decode picture
^Cv4l2-request: Timeout when waiting for media request
[h264 @ 0x81e8a0] Failed to end picture decode issue: 1 (operation failed).
[h264 @ 0x81e8a0] hardware accelerator failed to decode picture
v4l2-request: Timeout when waiting for media request
[h264 @ 0x860380] Failed to end picture decode issue: 1 (operation failed).
[h264 @ 0x860380] hardware accelerator failed to decode picture
v4l2-request: Unable to queue buffer: Invalid argument

is it possible to fix this? Thanks.

This library is out of date and won't work with recent kernel on any SoC.

@jernejsk It's working good on A33. Problem only on A23. Also what i should use instead of this library?

Interesting - this library wasn't updated for quiet some time. It was written at the very beginning of Cedrus development, more of a proof of concept. Interface used here doesn't correspond to latest mainline anymore. How did you manage to make it work? Which kernel did you use?

I'm afraid there is no replacement for this library. There are some forks which makes it work to some degree on newer kernels. Approach taken with gstreamer and ffmpeg is to use kernel interface directly, without intermediate library like this. However, that won't make it work with VLC and changes are not yet in any stable release of gstreamer nor ffmpeg (gstreamer changes are already in git master branch, while ffmpeg changes are WIP).

Regarding your issue - I see that BSP kernel uses a bit lower frequency on A23/A33, 300 MHz instead of 320 MHz. It could be that those additional 20 MHz make it unstable and A33 is a bit more tolerable. You can change that here: https://elixir.bootlin.com/linux/v5.14.1/source/drivers/staging/media/sunxi/cedrus/cedrus.c#L504

How did you manage to make it work? Which kernel did you use?

i'm using this pull request / fork https://github.com/noneucat/libva-v4l2-request/tree/fix-kernel-5.14 and kernel 5.14.0 from latest Linus tree https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git (commit 78e709522d2c012cb0daad2e668506637bffb7c2)

it working on A33 with VLC, not really good, but no timeouts

Regarding your issue - I see that BSP kernel uses a bit lower frequency on A23/A33, 300 MHz instead of 320 MHz. It could be that those additional 20 MHz make it unstable and A33 is a bit more tolerable. You can change that here: https://elixir.bootlin.com/linux/v5.14.1/source/drivers/staging/media/sunxi/cedrus/cedrus.c#L504

Like this?

diff --git a/arch/arm/boot/dts/sun8i-a23.dtsi b/arch/arm/boot/dts/sun8i-a23.dtsi
index a5e884a8b..54d8adecd 100644
--- a/arch/arm/boot/dts/sun8i-a23.dtsi
+++ b/arch/arm/boot/dts/sun8i-a23.dtsi
@@ -46,6 +46,17 @@

 / {
    soc {
+                video-codec@1c0e000 {
+                        compatible = "allwinner,sun8i-a33-video-engine";
+                        reg = <0x01c0e000 0x1000>;
+                        clocks = <&ccu CLK_BUS_VE>, <&ccu CLK_VE>,
+                                 <&ccu CLK_DRAM_VE>;
+                        clock-names = "ahb", "mod", "ram";
+                        resets = <&ccu RST_BUS_VE>;
+                        interrupts = <GIC_SPI 58 IRQ_TYPE_LEVEL_HIGH>;
+                        allwinner,sram = <&ve_sram 1>;
+                };
+
        codec: codec@1c22c00 {
            #sound-dai-cells = <0>;
            compatible = "allwinner,sun8i-a23-codec";
diff --git a/drivers/staging/media/sunxi/cedrus/cedrus.c b/drivers/staging/media/sunxi/cedrus/cedrus.c
index c0d005daf..5027341d9 100644
--- a/drivers/staging/media/sunxi/cedrus/cedrus.c
+++ b/drivers/staging/media/sunxi/cedrus/cedrus.c
@@ -501,7 +501,7 @@ static const struct cedrus_variant sun8i_a33_cedrus_variant = {
              CEDRUS_CAPABILITY_MPEG2_DEC |
              CEDRUS_CAPABILITY_H264_DEC |
              CEDRUS_CAPABILITY_VP8_DEC,
-   .mod_rate   = 320000000,
+   .mod_rate   = 300000000,
 };

 static const struct cedrus_variant sun8i_h3_cedrus_variant = {

Not working with vlc :(

[0079cbb0] avcodec decoder: Using v4l2-request for hardware decoding
v4l2-request: Timeout when waiting for media request
[h264 @ 0x7eb630] Failed to end picture decode issue: 1 (operation failed).
[h264 @ 0x7eb630] hardware accelerator failed to decode picture
v4l2-request: Timeout when waiting for media request

Sorry, no idea then. I checked both manuals and description of VE bits matches.

I'm afraid there is no replacement for this library. There are some forks which makes it work to some degree on newer kernels. Approach taken with gstreamer and ffmpeg is to use kernel interface directly, without intermediate library like this. However, that won't make it work with VLC and changes are not yet in any stable release of gstreamer nor ffmpeg (gstreamer changes are already in git master branch, while ffmpeg changes are WIP).

Ok let's try gstreamer.

GST_GL_API=gles2 GST_GL_PLATFORM=egl gst-launch-1.0 filesrc location=/home/user/test.mkv ! \
matroskademux ! queue ! h264parse ! v4l2slh264dec ! glimagesink

A33 - working A23 - got this error:

0:00:00.576111125  1483   0x4831a0 ERROR                     va gstvacaps.c:307:gst_va_create_raw_caps:<vadisplaydrm0> vaCreateConfig: the requested VAProfile is not supported
0:00:00.576490625  1483   0x4831a0 ERROR                     va gstvacaps.c:307:gst_va_create_raw_caps:<vadisplaydrm0> vaCreateConfig: the requested VAProfile is not supported
Setting pipeline to PAUSED ...
Xlib: sequence lost (0x1002e > 0x30) in reply type 0x0!
Xlib: sequence lost (0x10033 > 0x35) in reply type 0x0!
Xlib: sequence lost (0x10038 > 0x3a) in reply type 0x0!
Xlib: sequence lost (0x1003d > 0x3f) in reply type 0x0!
Pipeline is PREROLLING ...
Got context from element 'sink': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayX11\)\ gldisplayx11-0";
Redistribute latency...
ERROR: from element /GstPipeline:pipeline0/v4l2slh264dec:v4l2slh264dec0: Not enough memory to decode H264 stream.
Additional debug info:
../gst-build/subprojects/gst-plugins-bad/sys/v4l2codecs/gstv4l2codech264dec.c(913): gst_v4l2_codec_h264_dec_ensure_bitstream (): /GstPipeline:pipeline0/v4l2slh264dec:v4l2slh264dec0
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
0:00:04.679187127  1482   0x7cab38 ERROR            h264decoder gsth264decoder.c:980:gst_h264_decoder_start_current_picture:<v4l2slh264dec0> subclass does not want to start picture
0:00:04.679423169  1482   0x7cab38 ERROR            h264decoder gsth264decoder.c:1183:gst_h264_decoder_parse_slice:<v4l2slh264dec0> start picture failed

@jernejsk "Not enough memory to decode H264 stream" hmmm interesting. What does this mean? I have 'cma=256M' in kernel cmdline if this is important. Also i have 512MB ram on both devices (A33 and A23). I also tried 'cma=256M vmalloc=256M' - same error. after error gst-launch-1.0 stuck in Dl+ state.

$ ps axuf | grep gst-launch-1.0
root      1219  0.8  8.5 176288 40416 pts/0    Dl+  03:27   0:01      |                           \_ gst-launch-1.0 filesrc location=/home/user/test.mkv ! matroskademux ! queue ! h264parse ! v4l2slh264dec ! glimagesink

Also my device tree: https://gitlab.com/astroncia/sunxi-wexler-tab-a720/-/blob/master/patches/LINUX_0001-Add-Wexler-TAB-A720-support.patch - maybe something wrong here?

hm... there could be another driver issue. Can you try removing CEDRUS_CAPABILITY_UNTILED from here: https://elixir.bootlin.com/linux/v5.14.1/source/drivers/staging/media/sunxi/cedrus/cedrus.c#L500

It's quiet possible that A23 doesn't support it. Please test this with gstreamer git too. But you also have to have this bugfix: https://patchwork.linuxtv.org/project/linux-media/patch/20210819140009.158156-1-nicolas.dufresne@collabora.com/

@jernejsk

Please test this with gstreamer git too.

I am using gstreamer git. Kernel 5.15.0-rc1-00014-g316346243be6-dirty patch (including your bugfix):

diff --git a/drivers/staging/media/sunxi/cedrus/cedrus.c b/drivers/staging/media/sunxi/cedrus/cedrus.c
index c0d005daf..63b4b858b 100644
--- a/drivers/staging/media/sunxi/cedrus/cedrus.c
+++ b/drivers/staging/media/sunxi/cedrus/cedrus.c
@@ -497,11 +497,10 @@ static const struct cedrus_variant sun7i_a20_cedrus_variant = {
 };

 static const struct cedrus_variant sun8i_a33_cedrus_variant = {
-       .capabilities   = CEDRUS_CAPABILITY_UNTILED |
-                         CEDRUS_CAPABILITY_MPEG2_DEC |
+       .capabilities   = CEDRUS_CAPABILITY_MPEG2_DEC |
                          CEDRUS_CAPABILITY_H264_DEC |
                          CEDRUS_CAPABILITY_VP8_DEC,
-       .mod_rate       = 320000000,
+       .mod_rate       = 300000000,
 };

 static const struct cedrus_variant sun8i_h3_cedrus_variant = {
diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_video.c b/drivers/staging/media/sunxi/cedrus/cedrus_video.c
index c589fe9da..825af5fd3 100644
--- a/drivers/staging/media/sunxi/cedrus/cedrus_video.c
+++ b/drivers/staging/media/sunxi/cedrus/cedrus_video.c
@@ -135,7 +135,7 @@ void cedrus_prepare_format(struct v4l2_pix_format *pix_fmt)
                sizeimage = bytesperline * height;

                /* Chroma plane size. */
-               sizeimage += bytesperline * height / 2;
+               sizeimage += bytesperline * ALIGN(height, 64) / 2;

                break;

Error:

Setting pipeline to PAUSED ...
Xlib: sequence lost (0x1002e > 0x30) in reply type 0x0!
Xlib: sequence lost (0x10033 > 0x35) in reply type 0x0!
Xlib: sequence lost (0x10038 > 0x3a) in reply type 0x0!
Xlib: sequence lost (0x1003d > 0x3f) in reply type 0x0!
Pipeline is PREROLLING ...
Got context from element 'sink': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayX11\)\ gldisplayx11-0";
ERROR: from element /GstPipeline:pipeline0/v4l2slh264dec:v4l2slh264dec0: Unsupported bitdepth/chroma format
Additional debug info:
../gst-build/subprojects/gst-plugins-bad/sys/v4l2codecs/gstv4l2codech264dec.c(359): gst_v4l2_codec_h264_dec_negotiate (): /GstPipeline:pipeline0/v4l2slh264dec:v4l2slh264dec0:
No support for 1920x1088 8bit chroma IDC 1
ERROR: pipeline doesn't want to preroll.
0:00:01.291500793  1175   0x68dd38 ERROR     v4l2codecs-h264dec gstv4l2codech264dec.c:875:gst_v4l2_codec_h264_dec_new_sequence:<v4l2slh264dec0> Failed to negotiate with downstream
0:00:01.291708501  1175   0x68dd38 ERROR            h264decoder gsth264decoder.c:2293:gst_h264_decoder_process_sps:<v4l2slh264dec0> subclass does not want accept new sequence
Setting pipeline to NULL ...
Freeing pipeline ...

with GST_DEBUG="*:6":

0:00:05.051441711  1226   0x6e3738 DEBUG               GST_CAPS gstutils.c:3155:gst_pad_peer_query_caps:<v4l2slh264dec0:src> peer query returned EMPTY
0:00:05.051457919  1226   0x6e3770 DEBUG     matroskareadcommon matroska-read-common.c:1615:gst_matroska_read_common_parse_index_cuetrack:<matroskademux0:sink> CueTrack: 1
0:00:05.051508794  1226   0x6e3738 DEBUG     v4l2codecs-h264dec gstv4l2codech264dec.c:356:gst_v4l2_codec_h264_dec_negotiate:<v4l2slh264dec0> Peer supported formats: EMPTY
0:00:05.051514586  1226   0x6e3770 LOG                 ebmlread ebml-read.c:209:gst_ebml_peek_id_full:<matroskademux0> id 0xf1 at offset 0x50b346f of length 4, prefix 2
0:00:05.051588044  1226   0x6e3738 WARN      v4l2codecs-h264dec gstv4l2codech264dec.c:359:gst_v4l2_codec_h264_dec_negotiate:<v4l2slh264dec0> error: Unsupported bitdepth/chroma format
0:00:05.051592752  1226   0x6e3770 LOG                 ebmlread ebml-read.c:221:gst_ebml_peek_id_full:<matroskademux0> current br 0x7bd198; remaining 10
0:00:05.051636836  1226   0x6e3738 WARN      v4l2codecs-h264dec gstv4l2codech264dec.c:359:gst_v4l2_codec_h264_dec_negotiate:<v4l2slh264dec0> error: No support for 1920x1088 8bit chroma IDC 1

gst_v4l2_codec_h264_dec_negotiate:<v4l2slh264dec0> Peer supported formats: EMPTY

bootlin / libva-v4l2-request

A23 support #39