Help with RPI4 hw acceleration

[tranzmatt]

Hello,

Sorry if I'm asking an obvious question, but I have an rpi4 that needs to ingest multiple mjpeg video streams, re-transcode them to h264 to send to an external rtsp server. I figured I could do this with ffmpeg, but I don't seem to be able to get any hardware acceleration going and Googling around gives me a lot of stuff about omx and mmal, but that seems to have been removed, but I can't seem to get h264_v4l2m2m working either.

I saw this repo but am not sure how to build/compile for the rpi4. I'm more than familiar in building code, I'm just a bit lost with all the conflicting info I'm seeing out there about what configure switches to build with and how to call ffmpeg.

Can anyone provide some insight? I'm using the bullseye version of raspbian, but will use bookworm too.

Thanks

[jc-kynesim]

Lets start off with the easy questions: I assume you are on a Pi4? How many streams with what resolution and frame-rate? Are you trying to do this via command-line ffmpeg or are you writing code to talk to libavcodec?

[tranzmatt]

Yes, I have an RPi4. Ideally I want to do this on one running BalenaOS, but I first just want to get it working on a native version.

It could be up to 4 streams, currently at 30fps, but in addition to h264, I want to drop the frame rate to 10fps for bandwidth saving (I have zero control over the camera settings)

I'm using mediamtx, which basically makes a call to ffmpeg in its mediamtx.yml, such as this example, which isn't tailored for the pi.

paths:
  Camera_1:
    source: rtsp://192.168.1.100:8554/Streaming/channels/101
    runOnReady: >
      ffmpeg -i rtsp://localhost:$RTSP_PORT/$MTX_PATH
      -bufsize 1000k -fflags +igndts -c:v libx264 -rtsp_transport tcp -max_delay 5000000
      -f rtsp rtsp://my.camera-server.com:8554/Camera_1

[jc-kynesim]

So what resolution does your camera produce? I ask because the Pi H264 h/w encode tops out @ ~30fps 1920x1080. If the total from all streams exceeds that then you are doomed before you start. The encode block is pretty much the same one that was in the original Pi1 so it isn't very exciting by modern standards.

[tranzmatt]

Oof. One of the cameras under consideration is 2560x1440

So even if I ran multiple ffmpeg processes each @1920x1080, that would exceed the total allowed on the encoder?

[jc-kynesim]

Yup - you are stuffed - sorry - I'm pretty sure max encode res is 1920x1088 (maybe a bit bigger but not usefully) regardless of frame rate and yes the h/w doesn't care how many processes you run encode in it has got a hard overall limit.

[tranzmatt]

Well, this at least explains why I was making such little progress. Thanks for the help. Alas, the SW reencoding on a RPi5's CPU is just insane.

[tranzmatt]

Closed because I can't make use of the hw encoder as I want.