search

JeffyCN / FFmpeg

FFmpeg with rkmpp hwdec - (deprecated due to license issue, check gstreamer instead or https://github.com/nyanmisaka/ffmpeg-rockchip)
https://ffmpeg.org
Other
63 stars 19 forks source link

rkmpp failures(?) on rk3399 #19

Closed eigenbrot closed 1 year ago

eigenbrot commented 1 year ago

Hello,

I am trying to get either h264 or hvec hardware decoding working on a RockPi 4B. I have tried this using the official Radxa images of both Ubuntu Focal and Debian Buster. My current steps are:

  1. Make and install mpp from the latest commit of rockchip-linux's mpp repo. I make it with -DHAVE_DRM=ON as suggested by conventional wisdom.
  2. Build ffmpg from this repo and tag 4.4_20230105. I configure it with --enable-rkmpp --enable-version3 --enable-libdrm. I tried also to use --enable-librga, but I could not find those libraries anywhere.
  3. Everything builds fine and I can see the rkmpp decoders: 
    $ ffmpeg -decoders | grep mpp
ffmpeg version 4.4_20230105 Copyright (c) 2000-2021 the FFmpeg developers
built with gcc 8 (Debian 8.3.0-6)
configuration: --enable-rkmpp --enable-version3 --enable-libdrm
libavutil      56. 70.100 / 56. 70.100
libavcodec     58.134.100 / 58.134.100
libavformat    58. 76.100 / 58. 76.100
libavdevice    58. 13.100 / 58. 13.100
libavfilter     7.110.100 /  7.110.100
libswscale      5.  9.100 /  5.  9.100
libswresample   3.  9.100 /  3.  9.100
V..... h263_rkmpp           h263 (rkmpp) (codec h263)
V..... h264_rkmpp           h264 (rkmpp) (codec h264)
V..... hevc_rkmpp           hevc (rkmpp) (codec hevc)
V..... mpeg1_rkmpp          mpeg1 (rkmpp) (codec mpeg1video)
V..... mpeg2_rkmpp          mpeg2 (rkmpp) (codec mpeg2video)
V..... mpeg4_rkmpp          mpeg4 (rkmpp) (codec mpeg4)
V..... vp8_rkmpp            vp8 (rkmpp) (codec vp8)
V..... vp9_rkmpp            vp9 (rkmpp) (codec vp9)
  4. Now I try a test and here is where I think I see an issue: 
    $ ffmpeg -loglevel debug -benchmark -c:v hevc_rkmpp -i Tears.mkv -t 15 -f rawvideo /tmp/foo
ffmpeg version 4.4_20230105 Copyright (c) 2000-2021 the FFmpeg developers
built with gcc 8 (Debian 8.3.0-6)
configuration: --enable-rkmpp --enable-version3 --enable-libdrm
libavutil      56. 70.100 / 56. 70.100
libavcodec     58.134.100 / 58.134.100
libavformat    58. 76.100 / 58. 76.100
libavdevice    58. 13.100 / 58. 13.100
libavfilter     7.110.100 /  7.110.100
libswscale      5.  9.100 /  5.  9.100
libswresample   3.  9.100 /  3.  9.100
Splitting the commandline.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument 'debug'.
Reading option '-benchmark' ... matched as option 'benchmark' (add timings for benchmarking) with argument '1'.
Reading option '-c:v' ... matched as option 'c' (codec name) with argument 'hevc_rkmpp'.
Reading option '-i' ... matched as input url with argument 'Tears.mkv'.
Reading option '-t' ... matched as option 't' (record or transcode "duration" seconds of audio/video) with argument '15'.
Reading option '-f' ... matched as option 'f' (force format) with argument 'rawvideo'.
Reading option '/tmp/foo' ... matched as output url.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option loglevel (set logging level) with argument debug.
Applying option benchmark (add timings for benchmarking) with argument 1.
Successfully parsed a group of options.
Parsing a group of options: input url Tears.mkv.
Applying option c:v (codec name) with argument hevc_rkmpp.
Successfully parsed a group of options.
Opening an input file: Tears.mkv.
[NULL @ 0x559c4ce5d0] Opening 'Tears.mkv' for reading
[file @ 0x559c4cefb0] Setting default whitelist 'file,crypto,data'
[matroska,webm @ 0x559c4ce5d0] Format matroska,webm probed with size=2048 and score=100
st:0 removing common factor 1000000 from timebase
st:1 removing common factor 1000000 from timebase
[matroska,webm @ 0x559c4ce5d0] Before avformat_find_stream_info() pos: 5554 bytes read:32768 seeks:0 nb_streams:2
[hevc @ 0x559c4d1610] nal_unit_type: 32(VPS), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0x559c4d1610] Decoding VPS
[hevc @ 0x559c4d1610] Main profile bitstream
[hevc @ 0x559c4d1610] nal_unit_type: 33(SPS), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0x559c4d1610] Decoding SPS
[hevc @ 0x559c4d1610] Main profile bitstream
[hevc @ 0x559c4d1610] nal_unit_type: 34(PPS), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0x559c4d1610] Decoding PPS
[matroska,webm @ 0x559c4ce5d0] All info found
[matroska,webm @ 0x559c4ce5d0] After avformat_find_stream_info() pos: 19638 bytes read:32768 seeks:0 frames:20
Input #0, matroska,webm, from 'Tears.mkv':
Metadata:
encoder         : libebml v1.3.0 + libmatroska v1.4.0
creation_time   : 2013-09-04T21:15:18.000000Z
Duration: 00:12:14.06, start: 0.000000, bitrate: 1152 kb/s
Stream #0:0, 4, 1/1000: Video: hevc (Main), 1 reference frame, yuv420p(tv, left), 1280x720, 0/1, SAR 1:1 DAR 16:9, 24 fps, 24 tbr, 1k tbn, 1k tbc (default)
Stream #0:1, 16, 1/1000: Audio: aac (LC), 48000 Hz, 5.1, fltp (default)
Successfully opened the file.
Parsing a group of options: output url /tmp/foo.
Applying option t (record or transcode "duration" seconds of audio/video) with argument 15.
Applying option f (force format) with argument rawvideo.
Successfully parsed a group of options.
Opening an output file: /tmp/foo.
[file @ 0x559c4f6560] Setting default whitelist 'file,crypto,data'
Successfully opened the file.
[hevc_rkmpp @ 0x559c4d36d0] Format yuv420p chosen by get_format().
[hevc_rkmpp @ 0x559c4d36d0] Initializing RKMPP decoder.
[hevc_rkmpp @ 0x559c4d36d0] RKMPP decoder initialized successfully.
Stream mapping:
Stream #0:0 -> #0:0 (hevc (hevc_rkmpp) -> rawvideo (native))
Press [q] to stop, [?] for help
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[hevc_rkmpp @ 0x559c4d36d0] Wrote 346 bytes to decoder
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
Last message repeated 1 times
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
Last message repeated 16 times
[hevc_rkmpp @ 0x559c4d36d0] Wrote 48 bytes to decoder
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
Last message repeated 1 times
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[hevc_rkmpp @ 0x559c4d36d0] Wrote 44 bytes to decoder
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
Last message repeated 1 times
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[hevc_rkmpp @ 0x559c4d36d0] Wrote 44 bytes to decoder
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
Last message repeated 1 times
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Decoder noticed an info change (1280x720), format=0
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Wrote 43 bytes to decoder
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
Last message repeated 1 times
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
[hevc_rkmpp @ 0x559c4d36d0] Buffer full
[hevc_rkmpp @ 0x559c4d36d0] Received a frame.
[hevc_rkmpp @ 0x559c4d36d0] Doing slow software conversion
detected 6 logical cores
[graph 0 input from stream 0:0 @ 0x559c51d600] Setting 'video_size' to value '1280x720'
[graph 0 input from stream 0:0 @ 0x559c51d600] Setting 'pix_fmt' to value '0'
[graph 0 input from stream 0:0 @ 0x559c51d600] Setting 'time_base' to value '1/1000'
[graph 0 input from stream 0:0 @ 0x559c51d600] Setting 'pixel_aspect' to value '1/1'
[graph 0 input from stream 0:0 @ 0x559c51d600] Setting 'frame_rate' to value '24/1'
[graph 0 input from stream 0:0 @ 0x559c51d600] w:1280 h:720 pixfmt:yuv420p tb:1/1000 fr:24/1 sar:1/1
[AVFilterGraph @ 0x559c50f800] query_formats: 4 queried, 3 merged, 0 already done, 0 delayed
[hevc_rkmpp @ 0x559c4d36d0] Wrote 43 bytes to decoder
[hevc_rkmpp @ 0x559c4d36d0] Received a frame.
[hevc_rkmpp @ 0x559c4d36d0] Doing slow software conversion
[hevc_rkmpp @ 0x559c4d36d0] Received a frame.
[hevc_rkmpp @ 0x559c4d36d0] Doing slow software conversion
[hevc_rkmpp @ 0x559c4d36d0] Timeout getting decoded frame.
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[out_0_0 @ 0x559c51dc10] EOF on sink link out_0_0:default.
Output #0, rawvideo, to '/tmp/foo':
Metadata:
encoder         : Lavf58.76.100
Stream #0:0, 0, 1/24: Video: rawvideo, 1 reference frame (I420 / 0x30323449), yuv420p(left), 1280x720 (0x0) [SAR 1:1 DAR 16:9], 0/1, q=2-31, 265420 kb/s, 24 fps, 24 tbn (default)
Metadata:
  encoder         : Lavc58.134.100 rawvideo
No more output streams to write to, finishing.=00:00:00.00 bitrate=N/A speed=   0x    
[hevc_rkmpp @ 0x559c4d36d0] End of stream.
[hevc_rkmpp @ 0x559c4d36d0] Received a frame.
[hevc_rkmpp @ 0x559c4d36d0] Doing slow software conversion
frame=    0 fps=0.0 q=0.0 Lsize=       0kB time=00:00:00.00 bitrate=N/A speed=   0x    
video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Input file #0 (Tears.mkv):
Input stream #0:0 (video): 6 packets read (475 bytes); 4 frames decoded; 
Input stream #0:1 (audio): 16 packets read (13632 bytes); 
Total: 22 packets (14107 bytes) demuxed
Output file #0 (/tmp/foo):
Output stream #0:0 (video): 0 frames encoded; 0 packets muxed (0 bytes); 
Total: 0 packets (0 bytes) muxed
Output file is empty, nothing was encoded (check -ss / -t / -frames parameters if used)
bench: utime=0.156s stime=0.052s rtime=0.186s
4 frames successfully decoded, 0 decoding errors
bench: maxrss=20348kB
[AVIOContext @ 0x559c4f69d0] Statistics: 0 seeks, 0 writeouts
[AVIOContext @ 0x559c4d7090] Statistics: 32768 bytes read, 0 seeks

    In particular, I am suspicious of all the Timeout getting decoded frame and especially the [hevc_rkmpp @ 0x559c4d36d0] Doing slow software conversion. I get very similar results when trying an h264 file.

Any insight or advice would be greatly appreciated, thanks!

JeffyCN commented 1 year ago

try to enable librga with: https://github.com/JeffyCN/rockchip_mirrors/tree/linux-rga

eigenbrot commented 1 year ago

Thanks for the response. I was able to build the rga library you linked and then build FFmpeg with it. The errors now look different and have an RGA flavor:

$ ffmpeg -loglevel debug -benchmark -c:v hevc_rkmpp -i Tears.mkv -t 15 -f rawvideo /tmp/foo
ffmpeg version 4.4_20230105 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 8 (Debian 8.3.0-6)
  configuration: --enable-rkmpp --enable-version3 --enable-libdrm --enable-librga
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
Splitting the commandline.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument 'debug'.
Reading option '-benchmark' ... matched as option 'benchmark' (add timings for benchmarking) with argument '1'.
Reading option '-c:v' ... matched as option 'c' (codec name) with argument 'hevc_rkmpp'.
Reading option '-i' ... matched as input url with argument 'Tears.mkv'.
Reading option '-t' ... matched as option 't' (record or transcode "duration" seconds of audio/video) with argument '15'.
Reading option '-f' ... matched as option 'f' (force format) with argument 'rawvideo'.
Reading option '/tmp/foo' ... matched as output url.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option loglevel (set logging level) with argument debug.
Applying option benchmark (add timings for benchmarking) with argument 1.
Successfully parsed a group of options.
Parsing a group of options: input url Tears.mkv.
Applying option c:v (codec name) with argument hevc_rkmpp.
Successfully parsed a group of options.
Opening an input file: Tears.mkv.
[NULL @ 0x5581d86560] Opening 'Tears.mkv' for reading
[file @ 0x5581d86f40] Setting default whitelist 'file,crypto,data'
[matroska,webm @ 0x5581d86560] Format matroska,webm probed with size=2048 and score=100
st:0 removing common factor 1000000 from timebase
st:1 removing common factor 1000000 from timebase
[matroska,webm @ 0x5581d86560] Before avformat_find_stream_info() pos: 5554 bytes read:32768 seeks:0 nb_streams:2
[hevc @ 0x5581d89410] nal_unit_type: 32(VPS), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0x5581d89410] Decoding VPS
[hevc @ 0x5581d89410] Main profile bitstream
[hevc @ 0x5581d89410] nal_unit_type: 33(SPS), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0x5581d89410] Decoding SPS
[hevc @ 0x5581d89410] Main profile bitstream
[hevc @ 0x5581d89410] nal_unit_type: 34(PPS), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0x5581d89410] Decoding PPS
[matroska,webm @ 0x5581d86560] All info found
[matroska,webm @ 0x5581d86560] After avformat_find_stream_info() pos: 19638 bytes read:32768 seeks:0 frames:20
Input #0, matroska,webm, from 'Tears.mkv':
  Metadata:
    encoder         : libebml v1.3.0 + libmatroska v1.4.0
    creation_time   : 2013-09-04T21:15:18.000000Z
  Duration: 00:12:14.06, start: 0.000000, bitrate: 1152 kb/s
  Stream #0:0, 4, 1/1000: Video: hevc (Main), 1 reference frame, yuv420p(tv, left), 1280x720, 0/1, SAR 1:1 DAR 16:9, 24 fps, 24 tbr, 1k tbn, 1k tbc (default)
  Stream #0:1, 16, 1/1000: Audio: aac (LC), 48000 Hz, 5.1, fltp (default)
Successfully opened the file.
Parsing a group of options: output url /tmp/foo.
Applying option t (record or transcode "duration" seconds of audio/video) with argument 15.
Applying option f (force format) with argument rawvideo.
Successfully parsed a group of options.
Opening an output file: /tmp/foo.
[file @ 0x5581dae500] Setting default whitelist 'file,crypto,data'
Successfully opened the file.
[hevc_rkmpp @ 0x5581d8b650] Format yuv420p chosen by get_format().
[hevc_rkmpp @ 0x5581d8b650] Initializing RKMPP decoder.
[hevc_rkmpp @ 0x5581d8b650] RKMPP decoder initialized successfully.
Stream mapping:
  Stream #0:0 -> #0:0 (hevc (hevc_rkmpp) -> rawvideo (native))
Press [q] to stop, [?] for help
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[hevc_rkmpp @ 0x5581d8b650] Wrote 346 bytes to decoder
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
    Last message repeated 1 times
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
    Last message repeated 16 times
[hevc_rkmpp @ 0x5581d8b650] Wrote 48 bytes to decoder
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
    Last message repeated 1 times
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[hevc_rkmpp @ 0x5581d8b650] Wrote 44 bytes to decoder
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
    Last message repeated 1 times
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[hevc_rkmpp @ 0x5581d8b650] Wrote 44 bytes to decoder
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
    Last message repeated 1 times
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Decoder noticed an info change (1280x720), format=0
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Wrote 43 bytes to decoder
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
    Last message repeated 1 times
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
[hevc_rkmpp @ 0x5581d8b650] Buffer full
[hevc_rkmpp @ 0x5581d8b650] Received a frame.
failed to open RGA:No such file or directory.
rga_api version 1.3.1_[11] (1e2f0db build: 2023-05-05 08:13:45 base: )
Try to use uninit rgaCtx=(nil)
fd-vir-phy-hnd-format[19, (nil), (nil), (nil), 0]
rect[0, 0, 1280, 720, 1280, 720, 2560, 0]
f-blend-size-rotation-col-log-mmu[0, 0, 0, 0, 0, 0, 1]
fd-vir-phy-hnd-format[0, 0x7fb09b1010, (nil), (nil), 0]
rect[0, 0, 1280, 720, 1280, 768, 2816, 0]
f-blend-size-rotation-col-log-mmu[0, 0, 0, 0, 0, 0, 1]
This output the user patamaters when rga call blit fail
[hevc_rkmpp @ 0x5581d8b650] Doing slow software conversion
detected 6 logical cores
[graph 0 input from stream 0:0 @ 0x5581dd57b0] Setting 'video_size' to value '1280x720'
[graph 0 input from stream 0:0 @ 0x5581dd57b0] Setting 'pix_fmt' to value '0'
[graph 0 input from stream 0:0 @ 0x5581dd57b0] Setting 'time_base' to value '1/1000'
[graph 0 input from stream 0:0 @ 0x5581dd57b0] Setting 'pixel_aspect' to value '1/1'
[graph 0 input from stream 0:0 @ 0x5581dd57b0] Setting 'frame_rate' to value '24/1'
[graph 0 input from stream 0:0 @ 0x5581dd57b0] w:1280 h:720 pixfmt:yuv420p tb:1/1000 fr:24/1 sar:1/1
[AVFilterGraph @ 0x5581dc7810] query_formats: 4 queried, 3 merged, 0 already done, 0 delayed
[hevc_rkmpp @ 0x5581d8b650] Wrote 43 bytes to decoder
[hevc_rkmpp @ 0x5581d8b650] Received a frame.
Try to use uninit rgaCtx=(nil)
fd-vir-phy-hnd-format[24, (nil), (nil), (nil), 0]
rect[0, 0, 1280, 720, 1280, 720, 2560, 0]
f-blend-size-rotation-col-log-mmu[0, 0, 0, 0, 0, 0, 1]
fd-vir-phy-hnd-format[0, 0x7fb09b1010, (nil), (nil), 0]
rect[0, 0, 1280, 720, 1280, 768, 2816, 0]
f-blend-size-rotation-col-log-mmu[0, 0, 0, 0, 0, 0, 1]
This output the user patamaters when rga call blit fail
[hevc_rkmpp @ 0x5581d8b650] Doing slow software conversion
[hevc_rkmpp @ 0x5581d8b650] Received a frame.
Try to use uninit rgaCtx=(nil)
fd-vir-phy-hnd-format[23, (nil), (nil), (nil), 0]
rect[0, 0, 1280, 720, 1280, 720, 2560, 0]
f-blend-size-rotation-col-log-mmu[0, 0, 0, 0, 0, 0, 1]
fd-vir-phy-hnd-format[0, 0x7fb09b1010, (nil), (nil), 0]
rect[0, 0, 1280, 720, 1280, 768, 2816, 0]
f-blend-size-rotation-col-log-mmu[0, 0, 0, 0, 0, 0, 1]
This output the user patamaters when rga call blit fail
[hevc_rkmpp @ 0x5581d8b650] Doing slow software conversion
[hevc_rkmpp @ 0x5581d8b650] Timeout getting decoded frame.
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[out_0_0 @ 0x5581dc5ca0] EOF on sink link out_0_0:default.
Output #0, rawvideo, to '/tmp/foo':
  Metadata:
    encoder         : Lavf58.76.100
  Stream #0:0, 0, 1/24: Video: rawvideo, 1 reference frame (I420 / 0x30323449), yuv420p(left), 1280x720 (0x0) [SAR 1:1 DAR 16:9], 0/1, q=2-31, 265420 kb/s, 24 fps, 24 tbn (default)
    Metadata:
      encoder         : Lavc58.134.100 rawvideo
No more output streams to write to, finishing.=00:00:00.00 bitrate=N/A speed=   0x    
[hevc_rkmpp @ 0x5581d8b650] End of stream.
[hevc_rkmpp @ 0x5581d8b650] Received a frame.
Try to use uninit rgaCtx=(nil)
fd-vir-phy-hnd-format[25, (nil), (nil), (nil), 0]
rect[0, 0, 1280, 720, 1280, 720, 2560, 0]
f-blend-size-rotation-col-log-mmu[0, 0, 0, 0, 0, 0, 1]
fd-vir-phy-hnd-format[0, 0x7fb09b1010, (nil), (nil), 0]
rect[0, 0, 1280, 720, 1280, 768, 2816, 0]
f-blend-size-rotation-col-log-mmu[0, 0, 0, 0, 0, 0, 1]
This output the user patamaters when rga call blit fail
[hevc_rkmpp @ 0x5581d8b650] Doing slow software conversion
frame=    0 fps=0.0 q=0.0 Lsize=       0kB time=00:00:00.00 bitrate=N/A speed=   0x    
video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Input file #0 (Tears.mkv):
  Input stream #0:0 (video): 6 packets read (475 bytes); 4 frames decoded; 
  Input stream #0:1 (audio): 16 packets read (13632 bytes); 
  Total: 22 packets (14107 bytes) demuxed
Output file #0 (/tmp/foo):
  Output stream #0:0 (video): 0 frames encoded; 0 packets muxed (0 bytes); 
  Total: 0 packets (0 bytes) muxed
Output file is empty, nothing was encoded (check -ss / -t / -frames parameters if used)
bench: utime=0.144s stime=0.044s rtime=0.169s
4 frames successfully decoded, 0 decoding errors
bench: maxrss=24612kB
[AVIOContext @ 0x5581dae970] Statistics: 0 seeks, 0 writeouts
[AVIOContext @ 0x5581d8f050] Statistics: 32768 bytes read, 0 seeks

It looks like maybe RGA can't be found? This issue makes it look like maybe RGA is a module I need to build into the kernel? Are there any pre-made kernels that come with this?

Thanks

EDIT: For the record, I don't have a /dev/rga and I can't find anything RGA related with modprobe.

JeffyCN commented 1 year ago

the log looks like librga doesn't match rga kernel driver (there are multiple versions of those :( ) 3399 should use this: https://github.com/JeffyCN/mirrors/commits/linux-rga

and there's another version here to access rga driver without librga: https://github.com/JeffyCN/FFmpeg/issues/18#issuecomment-1528884921

eigenbrot commented 1 year ago

Ah, thanks for that link to the non-librga repo. That worked!