Feature request: Rockchip media acceleration opportunity

ShyneTurtle commented 3 months ago

I'm currently setting up Frigate on a RK3588 NAS, and i've come accross an FFMPEG instance ran by frigate that didn't use the internal media processing hardware (very high cpu usage compared other video processing tasks running on the machine). I suggest using MPP and RGA to transcode and scale respectively, see: https://github.com/nyanmisaka/ffmpeg-rockchip/wiki/Video-Transcode (i belive this is the FFMPEG fork used by the rockchip version of frigate).

I would love to contribute this myself but i'm not familiar enough with the codebase ^^'

Code subject to change: https://github.com/blakeblackshear/frigate/blob/a4eb435f1a9f64cde8a84357ea186de9072364d8/frigate/output/camera.py#L32-L57 and: https://github.com/blakeblackshear/frigate/blob/5c90f7dce7b24069ecfaeedb4be3f4892f380963/frigate/output/birdseye.py#L135-L160

Version: blakeblackshear/frigate:0.14.0-beta4-rk

hawkeye217 commented 3 months ago

CC @MarcA711

ShyneTurtle commented 3 months ago

(I might add that while i run the official build i checked Marc's fork and theses lines are the same so the issue might also be present)

MarcA711 commented 3 months ago

@ShyneTurtle thank you for this suggestion. We should probably add HW acceleration for this as well.

@hawkeye217 @NickM-27 is there already some implementation for hw specific FFmpeg presets? For Birdseye there are presets, but it seems they aren't used: https://github.com/blakeblackshear/frigate/blob/5c90f7dce7b24069ecfaeedb4be3f4892f380963/frigate/ffmpeg_presets.py#L105-L117

NickM-27 commented 3 months ago

The birdseye presets are used https://github.com/blakeblackshear/frigate/blob/5c90f7dce7b24069ecfaeedb4be3f4892f380963/docker/main/rootfs/usr/local/go2rtc/create_config.py#L12

github-actions[bot] commented 2 months ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

ShyneTurtle commented 2 months ago

This is annoying

MarcA711 commented 2 months ago

Sorry, I have this in mind but I didn't have time so far. I am currently very busy.

ShyneTurtle commented 2 months ago

Sorry, I have this in mind but I didn't have time so far. I am currently very busy.

Oh i was complaining about the bot, take your time bud ^^

a2nt commented 1 month ago

Looks like FFMpeg at the latest build ghcr.io/blakeblackshear/frigate:c17524b-rk doesn't support rkmpp anymore

MarcA711 commented 1 month ago

@a2nt Could you explain what you mean? I haven't tested this version yet, but it should use the same FFmpeg as before. Do you see any errors? What os are you using and what version (uname -a)?

NickM-27 commented 1 month ago

Yeah this is likely something with my PR, I will fix that.

a2nt commented 1 month ago

@a2nt Could you explain what you mean? I haven't tested this version yet, but it should use the same FFmpeg as before. Do you see any errors? What os are you using and what version (uname -a)?

well it's kinda simple ffmpeg is missing rkmpp

frigate  | 2024-09-17 13:31:10.295008656  [2024-09-17 13:31:10] ffmpeg.camera_1.detect         ERROR   : [vist#0:0/h264 @ 0xaaaad1f75c10] Unrecognized hwaccel: rkmpp.
frigate  | 2024-09-17 13:31:10.295387524  [2024-09-17 13:31:10] ffmpeg.camera_1.detect         ERROR   : [vist#0:0/h264 @ 0xaaaad1f75c10] Supported hwaccels: cuda drm opencl vulkan

so following config doesn't work any more:

ffmpeg:
  hwaccel_args: preset-rk-h264 # or preset-rk-h265
  output_args:
    record: preset-record-generic-audio-aac # preset-record-generic-audio-copy

ghcr.io/blakeblackshear/frigate:6bf2708-rk still has this issue

NickM-27 commented 1 month ago

go to frigate/api/stats and paste a couple of the ffmpeg commands

a2nt commented 1 month ago

frigate/api/stats

a2nt commented 1 month ago

go to frigate/api/stats and paste a couple of the ffmpeg commands

Actually weird question it can be checked easily.

Older version 0.14 supports rkmpp:

docker exec -it frigate bash
root@127:/opt/frigate# ffmpeg -hide_banner -hwaccels
Hardware acceleration methods:
cuda
drm
opencl
vulkan
rkmpp

while 0.15-0 does not:

root@127:/opt/frigate# /usr/lib/ffmpeg/6.0/bin/ffmpeg -hide_banner -hwaccels
Hardware acceleration methods:
cuda
drm
opencl
vulkan

NickM-27 commented 1 month ago

if you set the path to "ffmpeg" it will work. But the question is why which is not finding ffmpeg.

It would be really helpful if you could run

python3
import shutil
shutil.which("ffmpeg")

a2nt commented 1 month ago

hmm it shows nothing

if you set the path to "ffmpeg" it will work. But the question is why which is not finding ffmpeg.

It would be really helpful if you could run
python3
import shutil
shutil.which("ffmpeg")

NickM-27 commented 1 month ago

oh that's my fault, I made an incorrect assumption, will fix in a PR

a2nt commented 1 month ago

if you set the path to "ffmpeg" it will work. But the question is why which is not finding ffmpeg.

It would be really helpful if you could run
python3
import shutil
shutil.which("ffmpeg")

looks like need to add /usr/lib/ffmpeg to $PATH or link ffmpeg to /usr/bin or smth like that

blakeblackshear / frigate

Feature request: Rockchip media acceleration opportunity #12462