ffmpeg h264_omx encoding on RPi 4 running Ubuntu not working... where does the problem lie?

[mfreeborn]

Do let me know if this is not the correct place for this issue - I've been half way across the internet trying to solve this.

Describe the bug I'm trying to use the h264_omx encoder in ffmpeg to encode a video stream from the raspberry pi camera. I had no issue with this on my RPi 4 4GB running raspbian, but, for numerous reasons, I am now running Ubuntu 19.04.

Since changing operating system, I have been unable to get the h264_omx encoder to work.

I've simplified my set up as much as possible, I've got a random .gif from the internet and I am running this command:

ffmpeg -i giphy.gif -c:v h264_omx giphy.mp4 -y

The output is this:

ffmpeg version 4.1.4-1build2 Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.2.1-4ubuntu1)
  configuration: --prefix=/usr --extra-version=1build2 --toolchain=hardened --libdir=/usr/lib/aarch64-linux-gnu --incdir=/usr/include/aarch64-linux-gnu --arch=arm64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 22.100 / 56. 22.100
  libavcodec     58. 35.100 / 58. 35.100
  libavformat    58. 20.100 / 58. 20.100
  libavdevice    58.  5.100 / 58.  5.100
  libavfilter     7. 40.101 /  7. 40.101
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  3.100 /  5.  3.100
  libswresample   3.  3.100 /  3.  3.100
  libpostproc    55.  3.100 / 55.  3.100
Input #0, gif, from 'giphy.gif':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: gif, bgra, 500x375, 14.25 fps, 28.58 tbr, 100 tbn, 100 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (gif (native) -> h264 (h264_omx))
Press [q] to stop, [?] for help
[h264_omx @ 0xaaaac12e54a0] libOMX_Core.so not found
[h264_omx @ 0xaaaac12e54a0] libOmxCore.so not found
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
Conversion failed!

I can't find anything useful on the internet about libOMX_Core.so or libOmxCore.so.

ffmpeg -hwaccels produces the output:

Hardware acceleration methods:
vdpau
vaapi
drm

ffmpeg -encoders | grep omx produces:

V..... h264_omx             OpenMAX IL H.264 video encoder (codec h264)

It looks like it should be installed and working...

I found this thread and tried installing libomxil-bellagio-dev_ and libomxil-bellagio-bin to no avail.

I've tried recompiling ffmpeg but I get various errors such as C compiler test failed or ERROR: OpenMAX IL headers from raspberrypi/firmware not found

Software encoding works fine... can anyone help/point me in the right direction to get this working for hardware encoding please?

[popcornmix]

Are you using 32-bit or 64-bit ubuntu? openmax is not supported on 64-bit (and as a deprecated interface it never will be).

[mfreeborn]

Are you using 32-bit or 64-bit ubuntu? openmax is not supported on 64-bit (and as a deprecated interface it never will be).

Thank you for the quick reply.

64-bit is what I'm running as supposedly that has better performance.

Would it simply be a case of installing 32-bit Ubuntu and then it should work? i.e. this is a 32-bit vs 64-bit operating system problem rather than an Ubuntu vs Raspbian problem?

[popcornmix]

Would it simply be a case of installing 32-bit Ubuntu and then it should work? i.e. this is a 32-bit vs 64-bit operating system problem rather than an Ubuntu vs Raspbian problem?

Our testing is with raspbian and switching from 64-bit to 32-bit raspbian would have the effect of fixing the issue you are describing. I'd expect the same to occur with Ubuntu, but that is not something we typically test.

[mfreeborn]

Welp, backing up now - we'll see how it goes.

[6by9]

Assuming it's on a reasonably recent kernel, you should be able to use h264_v4l2m2m as your codec instead of h264_omx. That is again hardware assisted, and is supported in both 32 and 64 bit OSes.

[mfreeborn]

Assuming it's on a reasonably recent kernel, you should be able to use h264_v4l2m2m as your codec instead of h264_omx. That is again hardware assisted, and is supported in both 32 and 64 bit OSes.

Interesting, I did try that (I tried all h264_* just for the sake of it) and that one also did not work. Can't remember which error it threw, off the top of my head...

[mfreeborn]

Would it simply be a case of installing 32-bit Ubuntu and then it should work? i.e. this is a 32-bit vs 64-bit operating system problem rather than an Ubuntu vs Raspbian problem?

Our testing is with raspbian and switching from 64-bit to 32-bit raspbian would have the effect of fixing the issue you are describing. I'd expect the same to occur with Ubuntu, but that is not something we typically test.

So I've now installed Ubuntu 19.10 32-bit and the problem persists. I get the same output as in the first post.

I've also now tried using h264_v4l2m2m again and the error is as follows:

[h264_v4l2m2m @ 0xd68080] Could not find a valid device
[h264_v4l2m2m @ 0xd68080] can't configure encoder
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
Conversion failed!

[mfreeborn]

@popcornmix @6by9 Sorry to pester - any suggestions on what else I can try or where I can ask for help?

[marchom]

I have experienced a few issues with ffmpeg using the latest kernels on top of Ubuntu 18.04. Every once in a while getting the latest kernel would render the apt ffmpeg and any static linked ffmpeg to get Segfault. I have solved this by compiling ffmpeg from source. I have not pursued this further but perhaps there's a libc/translation layer somewhere from OS to the kernel which breaks compatibility...

And so I compile my own ffmpeg from source.

An added benefit is that you may disable x11 integration and a whole lot other things to slim the setup down if you are on a headless server.

https://potluru.wordpress.com/2016/06/26/compile-ffmpeg-for-raspberry-pi-3/

This is a bit outdated so most libraries are in apt now. This means you probably don't need to clone any other repo apart from ffmpeg.

 ~ $ sudo ffmpeg -i test.gif -c:v h264_omx giphy.mp4 -y
ffmpeg version git-2020-05-05-f90a48b Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 8 (Ubuntu/Linaro 8.4.0-1ubuntu1~18.04)
  configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc --enable-fontconfig --enable-gnutls --enable-gmp --enable-gray --enable-libass --enable-libfreetype --enable-libmp3lame --enable-libvorbis --enable-libx264 --enable-libx265 --enable-omx --enable-omx-rpi --arch=armel
  libavutil      56. 43.100 / 56. 43.100
  libavcodec     58. 82.100 / 58. 82.100
  libavformat    58. 42.102 / 58. 42.102
  libavdevice    58.  9.103 / 58.  9.103
  libavfilter     7. 80.100 /  7. 80.100
  libswscale      5.  6.101 /  5.  6.101
  libswresample   3.  6.100 /  3.  6.100
  libpostproc    55.  6.100 / 55.  6.100
Input #0, gif, from 'test.gif':
  Duration: 00:00:01.13, start: 0.000000, bitrate: 3166 kb/s
    Stream #0:0: Video: gif, bgra, 252x166, 23.58 fps, 29.97 tbr, 100 tbn, 100 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (gif (native) -> h264 (h264_omx))
Press [q] to stop, [?] for help
[h264_omx @ 0x21ec630] Using OMX.broadcom.video_encode
Output #0, mp4, to 'giphy.mp4':
  Metadata:
    encoder         : Lavf58.42.102
    Stream #0:0: Video: h264 (h264_omx) (avc1 / 0x31637661), yuv420p, 252x166, q=2-31, 200 kb/s, 29.97 fps, 30k tbn, 29.97 tbc
    Metadata:
      encoder         : Lavc58.82.100 h264_omx
frame=   34 fps=0.0 q=-0.0 Lsize=      26kB time=00:00:01.10 bitrate= 192.8kbits/s dup=7 drop=0 speed=9.67x
video:25kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 3.737733%

~ $ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.4 LTS
Release:        18.04
Codename:       bionic
~ $ uname -a
Linux server 4.19.118-v7l+ #1311 SMP Mon Apr 27 14:26:42 BST 2020 armv7l armv7l armv7l GNU/Linux

note the 'sudo' since I get some device access permission.

[mfreeborn]

Thanks @marchom - it was going so well until running configure for ffmpeg. I get the following error:

ERROR: OpenMAX IL headers from raspberrypi/firmware not found

[marchom]

Did you try:

sudo apt install libomxil-bellagio-dev

That's what fixed the header issues for me.

[mfreeborn]

Did you try:

sudo apt install libomxil-bellagio-dev

That's what fixed the header issues for me.

A bit further still.... I actually copied the contents of https://github.com/raspberrypi/firmware/tree/master/opt/vc/include/IL to /home/pi/ffmpeg/dependencies/output/include which got past the ./configure step and enabled me to run make and make install.

Now when running the newly compiled ffmpeg binary, I get the following error:

[h264_omx @ 0xaaaaeb032ef0] /opt/vc/lib/libbcm_host.so not found

I've tried copying thewhole contents of https://github.com/raspberrypi/firmware/tree/master/opt to the Raspberry Pi, but it still says it cannot find the libbcm_host.so despite me being able to see that it is located at that path...

[pelwell]

That libbcm_host.so is a 32-bit library, and [h264_omx @ 0xaaaaeb032ef0] (and earlier comments) makes me think you're trying to build a 64-bit ffmpeg. You'll have to try to build your own 64-bit libbcm_host.so as well.

[mfreeborn]

That libbcm_host.so is a 32-bit library, and [h264_omx @ 0xaaaaeb032ef0] (and earlier comments) makes me think you're trying to build a 64-bit ffmpeg. You'll have to try to build your own 64-bit libbcm_host.so as well.

Yes, this is on 64-bit Ubuntu. Are you able to point me in the right direction in terms of building my own 64-bit libbcm_host.so or, alternatively, is it easier/possible for me to build ffmpeg as a 32-bit binary (whilst still running 64-bit Ubuntu)

[pelwell]

Yes to the latter - a 64-bit kernel will happily run 32-bit applications, and 32-bit ffmpeg is tried-and-tested on Pi, so I would recommend that route.

[mfreeborn]

Yes to the latter - a 64-bit kernel will happily run 32-bit applications, and 32-bit ffmpeg is tried-and-tested on Pi, so I would recommend that route.

Well after a quick first Google... that looks like a can of worms. Would it be an option to compile ffmpeg as above on my (32-bit) RPi 3 and copy it over to my RPi 4?

[pelwell]

It should just work.

[mfreeborn]

It should just work.

Apologies for being so specific - it's a significant amount of time to compile and recompile these things - do you mean it should work if compiled on the RPi3 and then copied over?

[pelwell]

Yes - compile it on 32-bit. We are currently shipping 64-bit kernels to try along with our 32-bit-only Raspbian userland. I'm not aware of any problems with that configuration that wouldn't also affect a 64-bit userland (apart from running out of address space, etc.)

Unless you have to keep the Pi 4 up and running, why not do the compile on the Pi 4 in 32-bit mode to save time?

[marchom]

I should probably point out I compile it statically. This is my ./configure

./configure --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc --enable-fontconfig --enable-gnutls --enable-gmp --enable-gray --enable-libass --enable-libfreetype --enable-libmp3lame --enable-libvorbis --enable-libx264 --enable-libx265 --enable-omx --enable-omx-rpi --enable-h264_v4l2m2m --arch=armel

The missing ones that will complain are all lib...-dev packages so just trace them and you should be ok. I also should mention that I had a 64bit kernel and a 32 bit compiled binary and that screwed up things as well but give it a go. It was a while ago so maybe it's fixed now. The --arch=armel should give you 32 bit. you may need the cross compile aebi.

Alternatively I can send you my binary ~14Mb. Here:

https://send.firefox.com/download/9e78d3ba64ea12c3/#-DIh_IUteoauNDFvFWYFdg

[mfreeborn]

Thanks for all your help so far, I return well rested to try out a few more solutions.

With both the version I compiled on my RPi3 late last night as well as the static binary that was kindly shared with me, I was getting a file not found error, despite the binaries very obviously being present.

It turns out, in order to run 32-bit binaries on a 64-bit system I needed to follow these steps to install armhf support on 64-bit ubuntu.

Now when I try to run the shared statically linked binary from above..:

pi@pi4:~/Downloads$ ./ffmpeg
./ffmpeg: error while loading shared libraries: libass.so.9: cannot open shared object file: No such file or directory

Oh! Interrogate it with file and we get:

pi@pi4:~/Downloads$ file ./ffmpeg
./ffmpeg: ELF 32-bit LSB shared object, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 3.2.0, BuildID[sha1]=a2e504c37466199580473de2a1963650798656a8, stripped

It says it is dynamically linked!

So then I went and did more Googling, suggesting that in order to properly compile it statically it needs a slightly different ./configure - see here.

All of this has led to me compiling my own minimal ffmpeg using the following ./configure:

pi@pi-nestboxcam:~/FFmpeg $ ./configure --pkg-config-flags="--static" --prefix=/home/pi/ffmpeg/dependencies/output --enable-static --disable-shared --enable-gpl --enable-libx264 --enable-nonfree --enable-omx --enable-omx-rpi --extra-libs="-lx264 -lpthread -lm -ldl" --extra-cflags="-I/home/pi/ffmpeg/dependencies/output/include -static" --extra-ldflags="-L/home/pi/ffmpeg/dependencies/output/lib -static" --arch=armel

And we can see that it is now actually statically linked:

pi@pi4:~$ file ./ffmpeg
./ffmpeg: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), statically linked, for GNU/Linux 3.2.0, stripped

The binary now at least runs, but, terribly frustratingly, I get the same error as earlier - it is still trying to use /opt/vc/lib/libbcm_host.so:

pi@pi4:~$ ./ffmpeg -i test.gif -c:v h264_omx test.mp4 -y
ffmpeg version N-97666-g6fd61234d5 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 8 (Raspbian 8.3.0-6+rpi1)
  configuration: --pkg-config-flags=--static --prefix=/home/pi/ffmpeg/dependencies/output --enable-static --disable-shared --enable-gpl --enable-libx264 --enable-nonfree --enable-omx --enable-omx-rpi --extra-libs='-lx264 -lpthread -lm -ldl' --extra-cflags='-I/home/pi/ffmpeg/dependencies/output/include -static' --extra-ldflags='-L/home/pi/ffmpeg/dependencies/output/lib -static' --arch=armel
  libavutil      56. 43.100 / 56. 43.100
  libavcodec     58. 83.100 / 58. 83.100
  libavformat    58. 43.100 / 58. 43.100
  libavdevice    58.  9.103 / 58.  9.103
  libavfilter     7. 80.100 /  7. 80.100
  libswscale      5.  6.101 /  5.  6.101
  libswresample   3.  6.100 /  3.  6.100
  libpostproc    55.  6.100 / 55.  6.100
Input #0, gif, from 'test.gif':
  Duration: 00:00:00.56, start: 0.000000, bitrate: 7536 kb/s
    Stream #0:0: Video: gif, bgra, 500x375, 14.25 fps, 28.58 tbr, 100 tbn, 100 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (gif (native) -> h264 (h264_omx))
Press [q] to stop, [?] for help
[h264_omx @ 0x2d18800] /opt/vc/lib/libbcm_host.so not found
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
Conversion failed!

Here are the outputs for -encoders | grep h264 and -hwaccels, the latter of which is now empty:

pi@pi4:~$ ./ffmpeg -encoders | grep h264
<-- snip -->
 V..... libx264              libx264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (codec h264)
 V..... libx264rgb           libx264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 RGB (codec h264)
 V..... h264_omx             OpenMAX IL H.264 video encoder (codec h264)
 V..... h264_v4l2m2m         V4L2 mem2mem H.264 encoder wrapper (codec h264)

pi@pi4:~$ ./ffmpeg -hwaccels
<-- snip -->
Hardware acceleration methods:

[mfreeborn]

Also just to note, I tried copying both the /opt/vc/lib and hardfp/opt/vc/lib to see if that helped with the libbcm_host.so problem, but alas it did not.

[rayhughes34]

I am also trying to use hardware accelerated encoding on RPi 4 Ubuntu 64-bit. Compiled FFMpeg from source with sudo ./configure --arch=aarch64 --target-os=linux --enable-gpl --enable-libx264 --enable-libv4l2 --enable-libass —enable-nonfree, but getting this error when trying to encode a video:

[h264_v4l2m2m @ 0xaaaab2824570] Could not find a valid device
[h264_v4l2m2m @ 0xaaaab2824570] can't configure encoder
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height

[mfreeborn]

I am also trying to use hardware accelerated encoding on RPi 4 Ubuntu 64-bit. Compiled FFMpeg from source with sudo ./configure --arch=aarch64 --target-os=linux --enable-gpl --enable-libx264 --enable-libv4l2 --enable-libass —enable-nonfree, but getting this error when trying to encode a video:

[h264_v4l2m2m @ 0xaaaab2824570] Could not find a valid device
[h264_v4l2m2m @ 0xaaaab2824570] can't configure encoder
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height

My understanding of the v4l2m2m codec is limited, but as I understand it, it expects the input to be something like /dev/videoXX, which is a device that appears when e.g. a webcam is plugged in and the driver is enabled.

Just a quick update whilst I'm posting - I still haven't got hw acceleration working on Ubuntu, I've reverted back to Raspbian for the time being.

[marchom]

Did you resolve your Lib issue with the vc lib?

I remember adding it to my ld config so it knows to look for it in the right location. Creating a file in /etc/ld.so.conf.d/ with the content /opt/vc/lib should let your system find the library. I don't know it it might get into issues trying to link into it if it's a 32 bit binary and a 64 bit library.

For the /dev/videoXX i do have 3 interfaces there (10,11,12) with no additional hardware plugged in, so my guess is it uses them as encoder endpoints. That's why i needed to run with sudo otherwise user process doesn't get the permissions to use them.

[6by9]

/dev/video10 is the video decoder /dev/video11 is the video encoder /dev/video12 is the ISP for resize/format conversions.

All 3 are V4L2 memory to memory devices (https://www.kernel.org/doc/html/latest/media/uapi/v4l/dev-mem2mem.html), so each have an OUTPUT and CAPTURE queue, and each and every open of the device instantiates a new "conversion" session (ie you can have multiple independent decodes/encodes/conversions simultaneously).

If your permissions are wrong on those nodes then FFmpeg h264_v4l2m2m will fail (I'd expect root/video, and your user would normally be in the video group)

[marchom]

Yes, after adding my user to 'video' group:

user@server ~ $ ffmpeg -i ioEppjK.gif -c:v h264_v4l2m2m -pix_fmt yuv420p test.mp4 -y
ffmpeg version git-2020-05-10-fc99a24 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 8 (Ubuntu/Linaro 8.4.0-1ubuntu1~18.04)
  configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc --enable-fontconfig --enable-gnutls --enable-gmp --enable-gray --enable-libass --enable-libfreetype --enable-libmp3lame --enable-libvorbis --enable-libx264 --enable-libx265 --enable-omx --enable-omx-rpi --enable-libv4l2 --arch=armel
  libavutil      56. 44.100 / 56. 44.100
  libavcodec     58. 83.100 / 58. 83.100
  libavformat    58. 43.100 / 58. 43.100
  libavdevice    58.  9.103 / 58.  9.103
  libavfilter     7. 80.100 /  7. 80.100
  libswscale      5.  6.101 /  5.  6.101
  libswresample   3.  6.100 /  3.  6.100
  libpostproc    55.  6.100 / 55.  6.100
Input #0, gif, from 'ioEppjK.gif':
  Duration: 00:00:50.01, start: 0.000000, bitrate: 21895 kb/s
    Stream #0:0: Video: gif, bgra, 728x728 [SAR 64:64 DAR 1:1], 30 fps, 30 tbr, 100 tbn, 100 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (gif (native) -> h264 (h264_v4l2m2m))
Press [q] to stop, [?] for help
[h264_v4l2m2m @ 0x32e3c90] Using device /dev/video11
[h264_v4l2m2m @ 0x32e3c90] driver 'bcm2835-codec' on card 'bcm2835-codec-encode' in mplane mode
[h264_v4l2m2m @ 0x32e3c90] requesting formats: output=YU12 capture=H264
[h264_v4l2m2m @ 0x32e3c90] Failed to set gop size: Invalid argument
Output #0, mp4, to 'test.mp4':
  Metadata:
    encoder         : Lavf58.43.100
    Stream #0:0: Video: h264 (h264_v4l2m2m) (avc1 / 0x31637661), yuv420p, 728x728 [SAR 1:1 DAR 1:1], q=-1--1, 200 kb/s, 30 fps, 15360 tbn, 30 tbc
    Metadata:
      encoder         : Lavc58.83.100 h264_v4l2m2m
[mp4 @ 0x32e2840] Non-monotonous DTS in output stream 0:0; previous: 0, current: 0; changing to 1. This may result in incorrect timestamps in the output file.
frame=   14 fps=0.0 q=-0.0 size=       0kB time=00:00:00.40 bitratframe=   28 fps= 28 q=-0.0 size=       0kB time=00:00:00.86 bitratframe=   42 fps= 28 q=-0.0 size=       0kB time=00:00:01.33 bitratframe=   56 fps= 27 q=-0.0 size=       0kB time=00:00:01.80 bitratframe=   70 fps= 27 q=-0.0 size=       0kB time=00:00:02.26 bitratframe=   84 fps= 27 q=-0.0 size=       0kB time=00:00:02.73 bitratframe=   98 fps= 27 q=-0.0 size=     256kB time=00:00:03.20 bitratframe=  112 fps= 27 q=-0.0 size=     256kB time=00:00:03.66 bitratframe=  125 fps= 27 q=-0.0 size=     256kB time=00:00:04.10 bitratframe=  138 fps= 27 q=-0.0 size=     256kB time=00:00:04.53 bitratframe=  151 fps= 27 q=-0.0 size=     512kB time=00:00:04.96 bitratframe=  164 fps= 26 q=-0.0 size=     512kB time=00:00:05.40 bitratframe=  177 fps= 26 q=-0.0 size=     512kB time=00:00:05.83 bitratframe=  190 fps= 26 q=-0.0 size=     512kB time=00:00:06.26 bitratframe=  204 fps= 26 q=-0.0 size=     768kB time=00:00:06.73 bitratframe=  218 fps= 26 q=-0.0 size=     768kB time=00:00:07.20 bitratframe=  232 fps= 26 q=-0.0 size=     768kB time=00:00:07.66 bitratframe=  245 fps= 26 q=-0.0 size=     768kB time=00:00:08.10 bitratframe=  259 fps= 26 q=-0.0 size=     768kB time=00:00:08.56 bitratframe=  273 fps= 26 q=-0.0 size=    1024kB time=00:00:09.03 bitratframe=  287 fps= 26 q=-0.0 size=    1024kB time=00:00:09.50 bitratframe=  301 fps= 26 q=-0.0 size=    1024kB time=00:00:09.96 bitratframe=  314 fps= 26 q=-0.0 size=    1024kB time=00:00:10.40 bitratframe=  328 fps= 26 q=-0.0 size=    1280kB time=00:00:10.86 bitratframe=  342 fps= 26 q=-0.0 size=    1280kB time=00:00:11.33 bitratframe=  356 fps= 26 q=-0.0 size=    1280kB time=00:00:11.80 bitratframe=  370 fps= 26 q=-0.0 size=    1280kB time=00:00:12.26 bitratframe=  383 fps= 26 q=-0.0 size=    1536kB time=00:00:12.70 bitratframe=  396 fps= 26 q=-0.0 size=    1536kB time=00:00:13.13 bitratframe=  410 fps= 26 q=-0.0 size=    1536kB time=00:00:13.60 bitratframe=  424 fps= 26 q=-0.0 size=    1536kB time=00:00:14.06 bitratframe=  438 fps= 26 q=-0.0 size=    1792kB time=00:00:14.53 bitratframe=  452 fps= 26 q=-0.0 size=    1792kB time=00:00:15.00 bitratframe=  466 fps= 26 q=-0.0 size=    1792kB time=00:00:15.46 bitratframe=  479 fps= 26 q=-0.0 size=    1792kB time=00:00:15.90 bitratframe=  492 fps= 26 q=-0.0 size=    1792kB time=00:00:16.33 bitratframe=  505 fps= 26 q=-0.0 size=    2048kB time=00:00:16.76 bitratframe=  518 fps= 26 q=-0.0 size=    2048kB time=00:00:17.20 bitratframe=  531 fps= 26 q=-0.0 size=    2048kB time=00:00:17.63 bitratframe=  544 fps= 26 q=-0.0 size=    2048kB time=00:00:18.06 bitratframe=  557 fps= 26 q=-0.0 size=    2304kB time=00:00:18.50 bitratframe=  570 fps= 26 q=-0.0 size=    2304kB time=00:00:18.93 bitratframe=  583 fps= 26 q=-0.0 size=    2304kB time=00:00:19.36 bitratframe=  597 fps= 26 q=-0.0 size=    2304kB time=00:00:19.83 bitratframe=  611 fps= 26 q=-0.0 size=    2560kB time=00:00:20.30 bitratframe=  625 fps= 26 q=-0.0 size=    2560kB time=00:00:20.76 bitratframe=  638 fps= 26 q=-0.0 size=    2560kB time=00:00:21.20 bitratframe=  651 fps= 26 q=-0.0 size=    2560kB time=00:00:21.63 bitratframe=  664 fps= 26 q=-0.0 size=    2816kB time=00:00:22.06 bitratframe=  677 fps= 26 q=-0.0 size=    2816kB time=00:00:22.50 bitratframe=  691 fps= 26 q=-0.0 size=    2816kB time=00:00:22.96 bitratframe=  704 fps= 26 q=-0.0 size=    2816kB time=00:00:23.40 bitratframe=  717 fps= 26 q=-0.0 size=    2816kB time=00:00:23.83 bitratframe=  730 fps= 26 q=-0.0 size=    3072kB time=00:00:24.26 bitratframe=  743 fps= 26 q=-0.0 size=    3072kB time=00:00:24.70 bitratframe=  756 fps= 26 q=-0.0 size=    3072kB time=00:00:25.13 bitratframe=  770 fps= 26 q=-0.0 size=    3072kB time=00:00:25.60 bitratframe=  784 fps= 26 q=-0.0 size=    3328kB time=00:00:26.06 bitratframe=  797 fps= 26 q=-0.0 size=    3328kB time=00:00:26.50 bitratframe=  810 fps= 26 q=-0.0 size=    3328kB time=00:00:26.93 bitratframe=  823 fps= 26 q=-0.0 size=    3328kB time=00:00:27.36 bitratframe=  836 fps= 26 q=-0.0 size=    3328kB time=00:00:27.80 bitratframe=  850 fps= 26 q=-0.0 size=    3584kB time=00:00:28.26 bitratframe=  864 fps= 26 q=-0.0 size=    3584kB time=00:00:28.73 bitratframe=  877 fps= 26 q=-0.0 size=    3584kB time=00:00:29.16 bitratframe=  890 fps= 26 q=-0.0 size=    3840kB time=00:00:29.60 bitratframe=  903 fps= 26 q=-0.0 size=    3840kB time=00:00:30.03 bitratframe=  916 fps= 26 q=-0.0 size=    3840kB time=00:00:30.46 bitratframe=  930 fps= 26 q=-0.0 size=    3840kB time=00:00:30.93 bitratframe=  943 fps= 26 q=-0.0 size=    3840kB time=00:00:31.36 bitratframe=  956 fps= 26 q=-0.0 size=    4096kB time=00:00:31.80 bitratframe=  969 fps= 26 q=-0.0 size=    4096kB time=00:00:32.23 bitratframe=  982 fps= 26 q=-0.0 size=    4096kB time=00:00:32.66 bitratframe=  995 fps= 26 q=-0.0 size=    4352kB time=00:00:33.10 bitratframe= 1008 fps= 26 q=-0.0 size=    4352kB time=00:00:33.53 bitratframe= 1022 fps= 26 q=-0.0 size=    4352kB time=00:00:34.00 bitratframe= 1036 fps= 26 q=-0.0 size=    4352kB time=00:00:34.46 bitratframe= 1050 fps= 26 q=-0.0 size=    4608kB time=00:00:34.93 bitratframe= 1064 fps= 26 q=-0.0 size=    4608kB time=00:00:35.40 bitratframe= 1078 fps= 26 q=-0.0 size=    4608kB time=00:00:35.86 bitratframe= 1092 fps= 26 q=-0.0 size=    4608kB time=00:00:36.33 bitratframe= 1105 fps= 26 q=-0.0 size=    4608kB time=00:00:36.76 bitratframe= 1119 fps= 26 q=-0.0 size=    4608kB time=00:00:37.23 bitratframe= 1133 fps= 26 q=-0.0 size=    4608kB time=00:00:37.70 bitratframe= 1147 fps= 26 q=-0.0 size=    4864kB time=00:00:38.16 bitratframe= 1161 fps= 26 q=-0.0 size=    4864kB time=00:00:38.63 bitratframe= 1175 fps= 26 q=-0.0 size=    4864kB time=00:00:39.10 bitratframe= 1188 fps= 26 q=-0.0 size=    4864kB time=00:00:39.53 bitratframe= 1201 fps= 26 q=-0.0 size=    4864kB time=00:00:39.96 bitratframe= 1215 fps= 26 q=-0.0 size=    5120kB time=00:00:40.43 bitratframe= 1229 fps= 26 q=-0.0 size=    5120kB time=00:00:40.90 bitratframe= 1242 fps= 26 q=-0.0 size=    5120kB time=00:00:41.33 bitratframe= 1255 fps= 26 q=-0.0 size=    5120kB time=00:00:41.76 bitratframe= 1269 fps= 26 q=-0.0 size=    5376kB time=00:00:42.23 bitratframe= 1283 fps= 26 q=-0.0 size=    5376kB time=00:00:42.70 bitratframe= 1297 fps= 26 q=-0.0 size=    5376kB time=00:00:43.16 bitratframe= 1311 fps= 26 q=-0.0 size=    5376kB time=00:00:43.63 bitratframe= 1325 fps= 26 q=-0.0 size=    5376kB time=00:00:44.10 bitratframe= 1339 fps= 26 q=-0.0 size=    5632kB time=00:00:44.56 bitratframe= 1353 fps= 26 q=-0.0 size=    5632kB time=00:00:45.03 bitratframe= 1366 fps= 26 q=-0.0 size=    5632kB time=00:00:45.46 bitratframe= 1380 fps= 26 q=-0.0 size=    5632kB time=00:00:45.93 bitratframe= 1394 fps= 26 q=-0.0 size=    5888kB time=00:00:46.40 bitratframe= 1408 fps= 26 q=-0.0 size=    5888kB time=00:00:46.86 bitratframe= 1422 fps= 26 q=-0.0 size=    5888kB time=00:00:47.33 bitratframe= 1436 fps= 26 q=-0.0 size=    5888kB time=00:00:47.80 bitratframe= 1449 fps= 26 q=-0.0 size=    5888kB time=00:00:48.23 bitratframe= 1462 fps= 26 q=-0.0 size=    6144kB time=00:00:48.66 bitratframe= 1475 fps= 26 q=-0.0 size=    6144kB time=00:00:49.10 bitratframe= 1489 fps= 26 q=-0.0 size=    6144kB time=00:00:49.56 bitratframe= 1500 fps= 26 q=-0.0 Lsize=    6437kB time=00:00:49.96 bitrate=1055.4kbits/s speed=0.868x
video:6430kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.106382%

I had to also specify the input format as v4l2 was expecting something else.

This is on rpi4 with Ubuntu 18.04 32 bit on the 4.19.118-v7l+ 32 bit kernel. Compiled ffmpeg with:

./configure --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc --enable-fontconfig --enable-gnutls --enable-gmp --enable-gray --enable-libass --enable-libfreetype --enable-libmp3lame --enable-libvorbis --enable-libx264 --enable-libx265 --enable-omx --enable-omx-rpi --enable-libv4l2 --arch=armel

[rayhughes34]

v4l2-ctl --list-devices 10 returns the following for me:

Failed to open /dev/video10: No such file or directory

Note that I was running 64-bit Ubuntu. I also reverted to Raspbian for now.

[vishnukvmd]

I ran into issues similar to that of @mfreeborn and I resolved them after reverting to vanilla Raspbian and following this guide.

I was finally able to build ffmpeg with h264_omx.

ffmpeg -hide_banner -encoders | grep h264
 V..... libx264              libx264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (codec h264)
 V..... libx264rgb           libx264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 RGB (codec h264)
 V..... h264_omx             OpenMAX IL H.264 video encoder (codec h264)
 V..... h264_v4l2m2m         V4L2 mem2mem H.264 encoder wrapper (codec h264)

But when I use the h264_omx codec as follows, htop still shows the CPU maxing out (~400%).

ffmpeg -preset ultrafast -i VIDEO.MOV -profile:v baseline -vf scale=640:-1 -start_number 0 -hls_time 10 -hls_list_size 0 -f hls -c:v h264_omx index.m3u8

Also, ffmpeg -hide_banner -hwaccels returns an empty list of hardware acceleration methods.

Is this expected? If not, could someone please point me in the right direction? Thanks!

[ForsakenNGS]

I'm currently trying to get OMX working on my pi 4b running raspbian 32bit. I am NOT cross-compiling. I have libomxil-bellagio-dev, libomxil-bellagio0 and libomxil-bellagio-bin installed.

If I only use --enable-omx it compiles fine, but fails with:

[h264_omx @ 0x2e8a9a0] libOMX_Core.so not found
[h264_omx @ 0x2e8a9a0] libOmxCore.so not found

If I use both --enable-omx and --enable-omx-rpi the compile fails with:

ERROR: OpenMAX IL headers from raspberrypi/firmware not found

And indeed the OMX_Core.h header file provided by libomxil-bellagio-dev does not include any OMX_IndexConfigBrcmVideoRequestIFrame function/class/whatever.

Any ideas?

[outhud]

Same issue here as ForsakenNGS, but on Rpi3B, Raspbian 32-bit bullseye.

I was able to solve the ERROR: OpenMAX IL headers from raspberrypi/firmware not found error by updating the firmware first with sudo rpi-update.

But I still get the libOmxCore.so not found errors anyway, I don't find those files anywhere on my system, even after installing the libomxil-bellagio related packages.

[6by9]

libomxil-bellagio is the Bellagio reference implementation of OpenMAX IL. It is not useful on a PI.

OMX has been deprecated under Raspberry Pi OS Bullseye. Use h264_v4l2m2m.

[jpoles1]

h264_v4l2m2m

This is what worked for me in the end, thanks @6by9! Eg: ffmpeg ... -c:v h264_v4l2m2m

[VigibotDev]

Hello, " -c:v h264_v4l2m2m" do not want " -profile:v baseline" ! Any idea to get baseline profile ?

[6by9]

Hello, " -c:v h264_v4l2m2m" do not want " -profile:v baseline" ! Any idea to get baseline profile ?

https://ffmpeg.org/ffmpeg-codecs.html#Codec-Options profile integer (encoding,audio,video)

Set encoder codec profile. Default value is ‘unknown’. Encoder specific profiles are documented in the relevant encoder documentation.

https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/avcodec.h#L1587 FF_PROFILE_H264_BASELINE 66

-profile:v 66

[VigibotDev]

I just read this : https://forums.raspberrypi.com/viewtopic.php?p=1981727 Current perceived issues and limitations,

1. the ffmpeg -profile option only accepts an integer and rejects "-profile high"

And I tried dumbly 0, 1, 2 and more:) when I get your message, it's 66 !!!!!!!!!!!!! Thanks a lot !!!

[VigibotDev]

Outch 66 work, but I also need intra-headers inside my stream and :

" -flags:v +global_header", " -bsf:v dump_extra",

Look ignored. I RTFM :s

[6by9]

Sorry, these are really FFmpeg issues, not Pi kernel or firmware ones.

V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER is supported by the driver to insert header bytes before each I-frame, but FFmpeg doesn't allow setting of it.

A bsf (bitstream filter) entry should be independent of codec as it's another component inserted into the pipeline.

[VigibotDev]

look "-c:v h264_omx" don't work anymore on Bullseye :'(

I must try your patch to hardcode it inside ffmpeg...

inside libavcodec/v4l2_m2m_enc.c b/libavcodec/v4l2_m2m_enc.c I can add : v4l2_set_ext_ctrl(s, MPEG_CID(REPEAT_SEQ_HEADER), 1,"repeat parameter sets", 1);