h264_omx not working with Raspberry Pi

[eliabruni]

Hi!

I am trying to use the h264_omx encoding on the RPI, but I get this error:

[h264_omx @ 0x68500890] libOMX_Core.so not found
[h264_omx @ 0x68500890] libOmxCore.so not found
[libx264 @ 0x68502040] using cpu capabilities: ARMv6 NEON
[libx264 @ 0x68502040] profile Constrained Baseline, level 3.1, 4:2:0, 8-bit

However:

1) I build ffmpeg from source enabling omx; here are the steps i follow:

sudo apt-get remove ffmpeg
sudo apt-get install cmake libomxil-bellagio0 libomxil-bellagio-dev
git clone https://github.com/raspberrypi/userland.git
cd userland
./buildme
wget https://ffmpeg.org/releases/ffmpeg-6.0.tar.xz
tar -xf ffmpeg-6.0.tar.xz && cd ffmpeg-6.0
./configure --enable-omx --enable-omx-rpi
make && sudo make install

2) if i test the omx encoder from the terminal, on the same RPI

ffmpeg -i giphy.gif -c:v  h264_omx giphy.mp4 -y

it works. I get this:

ffmpeg version 6.0 Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 10 (Raspbian 10.2.1-6+rpi1)
  configuration: --enable-omx --enable-omx-rpi
  libavutil      58.  2.100 / 58.  2.100
  libavcodec     60.  3.100 / 60.  3.100
  libavformat    60.  3.100 / 60.  3.100
  libavdevice    60.  1.100 / 60.  1.100
  libavfilter     9.  3.100 /  9.  3.100
  libswscale      7.  1.100 /  7.  1.100
  libswresample   4. 10.100 /  4. 10.100
Input #0, gif, from 'giphy.gif':
  Duration: 00:00:02.19, start: 0.000000, bitrate: 6756 kb/s
  Stream #0:0: Video: gif, bgra, 800x600, 33.33 fps, 33.33 tbr, 100 tbn
Stream mapping:
  Stream #0:0 -> #0:0 (gif (native) -> h264 (h264_omx))
Press [q] to stop, [?] for help
[h264_omx @ 0x2012100] Using OMX.broadcom.video_encode
Output #0, mp4, to 'giphy.mp4':
  Metadata:
    encoder         : Lavf60.3.100
  Stream #0:0: Video: h264 (avc1 / 0x31637661), yuv420p(tv, progressive), 800x600, q=2-31, 200 kb/s, 33.33 fps, 12800 tbn
    Metadata:
      encoder         : Lavc60.3.100 h264_omx
frame=   31 fps=0.0 q=-0.0 Lsize=      24kB time=00:00:00.90 bitrate= 221.5kbits/s speed=1.05x

3) If I look for the missing files (libOMX_Core.so and libOmxCore.so), I cannot find them anywhere; however, I don't know if they should be somewhere in the first place or it is aiortc which is looking for the wrong files. Here is what I have:

ls /opt/vc/lib/
libbcm_host.so    libbrcmOpenVG.so  libdebug_sym.so        libEGL.so        libGLESv2_static.a  libmmal_components.so  libmmal_util.so       libOpenVG.so     libvcilcs.a  libWFC.so
libbrcmEGL.so     libbrcmWFC.so     libdebug_sym_static.a  libEGL_static.a  libkhrn_client.a    libmmal_core.so        libmmal_vc_client.so  libvchiq_arm.so  libvcos.so   pkgconfig
libbrcmGLESv2.so  libcontainers.so  libdtovl.so            libGLESv2.so     libkhrn_static.a    libmmal.so             libopenmaxil.so       libvchostif.a    libvcsm.so   plugins

Any help would be greatly appreciated!

[SaidGama]

I tried to use h264_omx, but I also couldn't get it to work. I conducted some research and it appears that h264_omx is deprecated in Bullseye and 64-bit systems. Try using h264_v4l2m2m, which is included by default in the ffmpeg package

[eliabruni]

Hey @SaidGama , thanks for the pointer. I indeed ended up using h264_v4l2m2m. I will also make a PR soon

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[neilyoung]

Sorry for re-animating this stale topic. I was following the suggestion and replaced h264_omx by h264_v4l2m2m. With my GStreamer pipelines I was using v4l2h264_enc as a replacement instead, but this seems to not work here.

h264_v4l2m2m is at least accepted, but there seems to be a little problem with the "baseline" option:

[h264_v4l2m2m @ 0x7f54003200] [Eval @ 0x7f756c0c88] Undefined constant or missing '(' in 'baseline'
[h264_v4l2m2m @ 0x7f54003200] Unable to parse option value "baseline"
[h264_v4l2m2m @ 0x7f54003200] Error setting option profile to value baseline.

Is there any information about the properties of this driver?

TIA

[SaidGama]

@neilyoung just remove baseline from the codec options like this: codec.options = { "level": "31", "tune": "zerolatency", }

[neilyoung]

Thanks for the quick answer. I just had to remove the "baseline" setting and did also remove "tune" :)

Regards

[neilyoung]

It seems to work fine, but just one thing bothers me a bit. I have added a trace right before create_encoder_context. Besides the gop_size error I'm a bit concerned about the fact, that the create_encoder_context is called over and over again during runtime. At least in the beginning for about 10-20 times. Is that related to the fact, that the bitrate is changing?

Instantiating h264_v4l2m2m...
[h264_v4l2m2m @ 0x7f6c003200] Using device /dev/video11
[h264_v4l2m2m @ 0x7f6c003200] driver 'bcm2835-codec' on card 'bcm2835-codec-encode' in mplane mode
[h264_v4l2m2m @ 0x7f6c003200] requesting formats: output=YU12 capture=H264
[h264_v4l2m2m @ 0x7f6c003200] Failed to set gop size: Invalid argument

[neilyoung]

But generally the h264_v4l2m2m is too lame, IMHO. I'm not able to achieve much more than 18-25 fps with a 720p input. This has been seen way better already on the same device (but with another codec). Too bad...