Update Rockchip presets

MarcA711 commented 4 months ago

Hey,

In https://github.com/AlexxIT/go2rtc/issues/768 I requested the implementation of presets for FFmpeg with Rockchip HW acceleration. These presets need to be updated, because the FFmpeg fork that most users used at this time was deprecated in favor of a new fork. Now, most Rockchip users use this fork from nyanmisaka. Here is the documentation on how to use the fork and here is a static version of this fork that I compiled.

In short, these presets allow transcoding and resizing a video using hardware:

// transcode to h264
ffmpeg -hwaccel rkmpp -hwaccel_output_format drm_prime -afbc rga -i input.mp4 -vf scale_rkrga=w={XXX}:h={XXX}:force_original_aspect_ratio=0 -c:v h264_rkmpp -g 50 -bf 0 -profile:v high -level:v 4.1 output.mp4
// transcode to hevc
ffmpeg -hwaccel rkmpp -hwaccel_output_format drm_prime -afbc rga -i input.mp4 -vf scale_rkrga=w={XXX}:h={XXX}:force_original_aspect_ratio=0 -c:v hevc_rkmpp -g 50 -bf 0 -profile:v main -level:v 5.1 output.mp4

I would really appreciate if you could update the presets and I can help testing.

nyanmisaka commented 4 months ago

https://github.com/AlexxIT/go2rtc/blob/5fa31fe4d6cf0e77562b755d52e8ed0165f89d25/internal/ffmpeg/ffmpeg_test.go#L163

FYI there's also a HW accelerated transpose/crop filter vpp_rkrga. The usage is similar to Intel QSV's vpp_qsv filter.

MarcA711 commented 1 month ago

Hey @AlexxIT , I just wanted to ask if the presets will be updated soon? It would be very nice to see them in the next version of Frigate. However, don't feel pressured. If you don't have time right now, that's fine too! In any case, thanks for your help.

AlexxIT commented 1 month ago

Hi. It's a big task for me. I need to replicate the entire environment so I don't have to make edits blindly. The task is in the plans, but I'm doing other tasks in the meanwhile https://github.com/AlexxIT/go2rtc/issues/assigned/AlexxIT

MarcA711 commented 1 month ago

Thank you for the clarification! Let me know, if I can help you somehow.

bugui2007 commented 3 weeks ago

Hi guys, I'm using frigate 14 beta2 and I noticed that the predefined functions like #rotate=90 and #audio=xxx were not working on my rockchip 3588 (it's an Orange PI 5 plus).

Is there any relationship between audio transcoding and this issue here?

Thanks in advance, Thank you so much. PS: Congratulations on this great project. Go2RTC is the perfect knife for streaming video!

MarcA711 commented 3 weeks ago

AFAIK, these options shouldn't cause any issues if you don't use #hardware.

AlexxIT commented 2 weeks ago

I don't have problems with transcoding. But have problems with scale on my Rockchip. Maybe because not best linux from LubanCat developers...

Maybe someone can help me with downloading latest version. Because I don't have Baidu account. I have Thinkcore Pi TP-0W / LubanCat Zero W / RK3566.

Baidu Cloud Link: https://pan.baidu.com/s/19t8AZV9SYTdjn2uObBiSGA Extraction code: hslu 3-Linux > rk356x > Ubuntu > Universal > 22.04 > 20240320 > rk3566 > ubuntu22.04-lite

cat@lubancat:~$ ./ffmpeg -hide_banner -hwaccel rkmpp -hwaccel_output_format drm_prime -re -i bbb5.mp4 -c:v h264_rkmpp -g 50 -bf 0 -profile:v high -level:v 4.1 -an -vf "vpp_rkrga=640:-1" -y out.mp4
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'bbb5.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf60.16.100
  Duration: 00:00:05.02, start: 0.000000, bitrate: 2606 kb/s
  Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], 2481 kb/s, 24 fps, 24 tbr, 24k tbn (default)
    Metadata:
      handler_name    : (C) 2007 Google Inc. v08.13.2007.
      vendor_id       : [0][0][0][0]
  Stream #0:1[0x2](und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 125 kb/s (default)
    Metadata:
      handler_name    : (C) 2007 Google Inc. v08.13.2007.
      vendor_id       : [0][0][0][0]
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (h264_rkmpp) -> h264 (h264_rkmpp))
Press [q] to stop, [?] for help
librga fail to get driver version! Compatibility mode will be enabled.

65 im2d_rga_impl rga_version_below_minimun_range_user_driver(322): The driver may be compatible, but it is best to update the driver to version 1.2.4. You can try to update the SDK or update the <SDK>/kernel/drivers/video/rockchip/rga3 directory individually. current version: librga 1.10.0, driver .
rga_api version 1.10.0_[8]
[Parsed_vpp_rkrga_0 @ 0x558edbb130] RGA async blit returned invalid fence_fd: 0
[vf#0:0 @ 0x558ee1c650] Error while filtering: Generic error in an external library
Failed to inject frame into filter network: Generic error in an external library
Error while filtering: Generic error in an external library
[out#0/mp4 @ 0x558edcafa0] Nothing was written into output file, because at least one of its streams received no packets.
frame=    0 fps=0.0 q=0.0 Lsize=       0kB time=N/A bitrate=N/A speed=N/A    
Conversion failed!

MarcA711 commented 2 weeks ago

Sorry, I can't help with downloading the newer image.

Maybe sudo helps, if your issue is due to insufficient permission. But it seems your issue is related to outdated drivers.

However, I can confirm that the above command works.

AlexxIT commented 2 weeks ago

LubanCat also build ffmpeg for their boards. Maybe something useful there: https://github.com/LubanCat/buildroot/tree/rk356x/package/ffmpeg

MarcA711 commented 2 weeks ago

This is pretty old and based on the original FFmpeg fork from rockchip, that was discontinued due to license issues. I don't think it works on up to date systems.

Currently, most users use this fork from nyanmisaka: https://github.com/nyanmisaka/ffmpeg-rockchip It works best in my opinion and is actively meintained. So go2rtc should implement presets for this fork.

nyanmisaka commented 2 weeks ago

You can try commenting out these lines in FFmpeg to avoid the check for a valid value for the kernel returned fence_fd. It might work, but ultimately you're better off updating your RGA kernel drivers to the latest and enabling Multi-RGA and Async-RGA in defconfig, which will help performance a lot.

https://github.com/nyanmisaka/ffmpeg-rockchip/blob/9efe5bcff082d6538702d01c9b7126f40da27361/libavfilter/rkrga_common.c#L1109-L1114

AlexxIT commented 2 weeks ago

Thanks. Building custom FFmpeg and updating RGA kernel drivers are too complicated things for me :)

I can run some docker container if there are pre-made ones with the right contents. Or I'll wait for someone to help me download the latest Linux from Baidu for my test board.

AlexxIT commented 2 weeks ago

I have tested latest Ubuntu from LubanCat developers. Same error. Also I can't run docker on this board. Don't know why. Just a waste of money...

AlexxIT commented 2 weeks ago

@MarcA711 what about Frigate's docker container? If I can run it on this board, then theoretically it should have correct version of library?

MarcA711 commented 2 weeks ago

I don't think it will work. Your problem is outdated kernel drivers and docker containers use the same kernel as the host.

Is there a lot that has to be done to update the presets? Maybe I am able to implement the presets if you point me to the right files. It should be similar to nvidia for example. However, I don't know Go.

Alternatively, maybe I could give you access to a docker container on my board using https://tmate.io/. Do you need docker for your tests or do you just want to run ffmpeg commands?

AlexxIT commented 2 weeks ago

go2rtc doesn't just use presets. It rewrites the full ffmpeg command from firmware to hardware version. By modifying inputs and filters. Nvidia not best example. VAAPI has better API.

https://github.com/AlexxIT/go2rtc/blob/master/internal/ffmpeg/hardware/hardware.go

MarcA711 commented 2 weeks ago

Thank you, I will have a look. Maybe I find out how to do it.

And for your board, you can maybe ask here for an image: https://github.com/Joshua-Riek/ubuntu-rockchip He already added support for other lubancat boards and it seems to be now big deal: https://github.com/Joshua-Riek/ubuntu-rockchip/issues/327. With an up to date base os, this will be a powerful board I guess.

AlexxIT commented 2 weeks ago

Thanks. I think RockChip support is really important in go2rtc. I bought this board specifically for that purpose. It was pretty cheap. I didn't think there would be so many problems with it.

MarcA711 commented 2 weeks ago

Yes, I think the rockchip platform is important. Rockchip is perhaps the most important platform after Raspberry Pi.

I was able to setup a dev environment on my board and hope to be able to implement the presets myself.

AlexxIT commented 2 weeks ago

It's not necessary to run go2rtc or dev on board. You can just fix code on any PC, run tests, and check output string after parseArgs on board's console. But change input and output to files.

https://github.com/AlexxIT/go2rtc/blob/ba3485560208c4f6cc9e7c36d645ca5e3b9f3d8b/internal/ffmpeg/ffmpeg_test.go#L195-L205

MarcA711 commented 2 weeks ago

This helps a lot. Thank you!

AlexxIT / go2rtc

Update Rockchip presets #979