search

JeffyCN / FFmpeg

FFmpeg with rkmpp hwdec - (deprecated due to license issue, check gstreamer instead or https://github.com/nyanmisaka/ffmpeg-rockchip)
https://ffmpeg.org
Other
60 stars 18 forks source link

Question) RockPi 4c+ ffmpeg hwdecode cur_dts is invalid #15

Closed salvobellino95 closed 1 year ago

salvobellino95 commented 1 year ago

Hi, thank you for your work! Im trying to build ffmpeg to support hw decode on a Rock Pi 4c + with legacy Debian vendor recommended OS. Just found this repo linked in many forum threads.

uname -r 4.4.194-11-rk3399-rockchip-g1bb08d49cc40

I cloned this repo, installed rockchip mpp from the official repo and builded this ffmpeg with enable-rkmpp but it does not work. Do I missed something?

git clone -b release https://github.com/rockchip-linux/mpp.git
sudo cmake -DRKPLATFORM=ON -DHAVE_DRM=ON
sudo make install -j6

git clone https://github.com/JeffyCN/FFmpeg
cd FFmpeg
sudo ./configure --enable-rkmpp --enable-librga --enable-version3 --enable-libdrm
sudo make

wget https://test-videos.co.uk/vids/bigbuckbunny/mp4/h264/1080/Big_Buck_Bunny_1080_10s_30MB.mp4

sudo ./ffmpeg -benchmark -loglevel debug -hwaccel drm -hwaccel_device /dev/dri/renderD128 -c:v h264_rkmpp -i Big_Buck_Bunny_1080_10s_30MB.mp4 -t 15 -pix_fmt yuv420p -f rawvideo out4.yu

But It does not work, many cur_dts is invalid errors and "Timeout getting decoded frame". Can you help me?

Successfully opened the file.
Parsing a group of options: output url out4.yu.
Applying option t (record or transcode "duration" seconds of audio/video) with argument 15.
Applying option pix_fmt (set pixel format) with argument yuv420p.
Applying option f (force format) with argument rawvideo.
Successfully parsed a group of options.
Opening an output file: out4.yu.
File 'out4.yu' already exists. Overwrite? [y/N] y
[file @ 0x55b2da90f0] Setting default whitelist 'file,crypto,data'
Successfully opened the file.
[AVHWDeviceContext @ 0x55b2bb0990] Opened DRM device /dev/dri/renderD128: driver rockchip version 1.0.0.
[h264_rkmpp @ 0x55b2bda7d0] Format yuv420p chosen by get_format().
[h264_rkmpp @ 0x55b2bda7d0] Initializing RKMPP decoder.
[h264_rkmpp @ 0x55b2bda7d0] RKMPP decoder initialized successfully.
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (h264_rkmpp) -> rawvideo (native))
Press [q] to stop, [?] for help
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[h264_rkmpp @ 0x55b2bda7d0] Wrote 1157871 bytes to decoder
[h264_rkmpp @ 0x55b2bda7d0] Timeout getting decoded frame.
    Last message repeated 1 times
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[h264_rkmpp @ 0x55b2bda7d0] Wrote 312947 bytes to decoder
[h264_rkmpp @ 0x55b2bda7d0] Timeout getting decoded frame.
    Last message repeated 1 times
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[h264_rkmpp @ 0x55b2bda7d0] Wrote 46490 bytes to decoder
[h264_rkmpp @ 0x55b2bda7d0] Timeout getting decoded frame.
    Last message repeated 1 times
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[h264_rkmpp @ 0x55b2bda7d0] Wrote 9841 bytes to decoder
[h264_rkmpp @ 0x55b2bda7d0] Decoder noticed an info change (1920x1080), format=0
[h264_rkmpp @ 0x55b2bda7d0] Timeout getting decoded frame.
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[h264_rkmpp @ 0x55b2bda7d0] Buffer full
[h264_rkmpp @ 0x55b2bda7d0] Timeout getting decoded frame.
[h264_rkmpp @ 0x55b2bda7d0] Buffer full
[h264_rkmpp @ 0x55b2bda7d0] Timeout getting decoded frame.
[h264_rkmpp @ 0x55b2bda7d0] Buffer full
[h264_rkmpp @ 0x55b2bda7d0] Timeout getting decoded frame.
[h264_rkmpp @ 0x55b2bda7d0] Buffer full
[h264_rkmpp @ 0x55b2bda7d0] Timeout getting decoded frame.
[h264_rkmpp @ 0x55b2bda7d0] Buffer full
[h264_rkmpp @ 0x55b2bda7d0] Timeout getting decoded frame.
[h264_rkmpp @ 0x55b2bda7d0] Buffer full
[h264_rkmpp @ 0x55b2bda7d0] Timeout getting decoded frame.
[h264_rkmpp @ 0x55b2bda7d0] Buffer full
[h264_rkmpp @ 0x55b2bda7d0] Timeout getting decoded frame.
[h264_rkmpp @ 0x55b2bda7d0] Buffer full
[h264_rkmpp @ 0x55b2bda7d0] Timeout getting decoded frame.
[h264_rkmpp @ 0x55b2bda7d0] Buffer full
[h264_rkmpp @ 0x55b2bda7d0] Timeout getting decoded frame.
[h264_rkmpp @ 0x55b2bda7d0] Buffer full
[h264_rkmpp @ 0x55b2bda7d0] Timeout getting decoded frame.
[h264_rkmpp @ 0x55b2bda7d0] Buffer full
[h264_rkmpp @ 0x55b2bda7d0] Timeout getting decoded frame.
[h264_rkmpp @ 0x55b2bda7d0] Buffer full
[h264_rkmpp @ 0x55b2bda7d0] Timeout getting decoded frame.
[h264_rkmpp @ 0x55b2bda7d0] Buffer full
[h264_rkmpp @ 0x55b2bda7d0] Timeout getting decoded frame.
[h264_rkmpp @ 0x55b2bda7d0] Buffer full
[h264_rkmpp @ 0x55b2bda7d0] Timeout getting decoded frame.
[h264_rkmpp @ 0x55b2bda7d0] Buffer full
[h264_rkmpp @ 0x55b2bda7d0] Timeout getting decoded frame.
[h264_rkmpp @ 0x55b2bda7d0] Buffer full
[h264_rkmpp @ 0x55b2bda7d0] Timeout getting decoded frame.
[h264_rkmpp @ 0x55b2bda7d0] Buffer full
[h264_rkmpp @ 0x55b2bda7d0] Timeout getting decoded frame.
salvobellino95 commented 1 year ago 
rock@rock-4c-plus:~/mpp$ sudo ./osal/test/mpp_platform_test
mpp[22812]: mpp_plat_test: chip name: radxa,rock-4c-plus rockchip,rk3399
mpp[22812]: mpp_plat_test:
mpp[22812]: mpp_plat_test: chip vcodec type 00040202
mpp[22812]: mpp_plat_test: found vpu2 codec
mpp[22812]: mpp_plat_test: found rkvdec decoder
mpp[22812]: mpp_plat_test: found vpu2 encoder
mpp[22812]: mpp_plat_test:
mpp[22812]: mpp_plat_test: start probing decoder device name:
mpp[22812]: mpp_plat_test: H.264 decoder: /dev/rkvdec
mpp[22812]: mpp_plat_test: H.265 decoder: /dev/rkvdec
mpp[22812]: mpp_plat_test: MJPEG decoder: /dev/vpu_service
mpp[22812]: mpp_plat_test: VP9   decoder: /dev/rkvdec
mpp[22812]: mpp_plat_test: avs   decoder: /dev/vpu_service
mpp[22812]: mpp_plat_test:
mpp[22812]: mpp_plat_test: start probing encoder device name:
mpp[22812]: mpp_plat_test: H.264 encoder: /dev/vpu_service
mpp[22812]: mpp_plat_test: H.265 encoder: /dev/vpu_service
mpp[22812]: mpp_plat_test: MJPEG encoder: /dev/vpu_service
mpp[22812]: mpp_plat_test: mpp platform test done
JeffyCN commented 1 year ago

the ffmpeg's rkmppdec is just a wrapper of mpp.

so maybe try the mpp's dec tests firstly to make sure that mpp is ok.

the mpp tests' input file should be raw h264 data or maybe try gstreamer rkmpp

cpmbailey commented 1 year ago

gstreamer is hardware accelerated on the 4c+. However I would prefer to use mpv and am encountering the same issue with ffmpeg.

salvobellino95 commented 1 year ago

I have to use FFmpeg because the final goal is to install Frigate NVR which does not support gstreamer, I tried directly mpp decode test and it does not works reporting Segmentation fault, I am investigating the root cause

salvobellino95 commented 1 year ago

@cpmbailey Maybe we can also search for a way to redirect commands for ffmpeg to gstreamer, with a fake ffmpeg that simply does this redirect, seems a crazy idea but it could works, I will investigate, do you have a command to test the decode using gstreamer on a sample h264 file?

cpmbailey commented 1 year ago

gst-launch-1.0 filesrc location=/home/rock/test.h264 ! h264parse ! avdec_h264 ! videoconvert ! ximagesink

salvobellino95 commented 1 year ago

Here I think that you are not using hwaccel, as I know you should use mppvideodec which is the decoder added by gstreamer-rockchip plugin, if you use avdec_h264 you are doing sw decode

Moreover I do not found gstreamer installed in the Raxda official build, do you installed it manually?

salvobellino95 commented 1 year ago

@cpmbailey I found that my test file was broken, but testing with gstreamer it can play the video 

sudo wget https://github.com/adafruit/pi_hello_video/raw/master/hello_video/test.h264
gst-launch-1.0 filesrc location=./test.h264 ! h264parse ! mppvideodec ! kmssink

This play the video (does not work via SSH). Here I am using mppvideodec @JeffyCN how this can work if mpi_dec_test reports Segmentation fault and apparently does not work? There are many strange things here to understand

cpmbailey commented 1 year ago

Sorry, copied the wrong one from my history but looks like you got it working.

cpmbailey commented 1 year ago

mpi_dec_test is working for me now, operator error

cpmbailey commented 1 year ago

I still get some timeout getting decoded frame messages but this seems to complete for me now. Are you sure that ffmpeg is using the correct librockchip_mpp.so.1?

salvobellino95 commented 1 year ago

@cpmbailey are you using the builtin rockchip mpp or you reinstalled it?

Are you sure that ffmpeg is using the correct librockchip_mpp.so.1?

How can I check this?

cpmbailey commented 1 year ago

I rebuilt it from the develop branch (not sure if that is necessary) and then copied it over the one that ffmpeg is linking against from "ldd fmpeg".

Have managed to get mpv working and hardware accelerated now

salvobellino95 commented 1 year ago

are you using FFMPEG from this repo? do you remember the branch? I will try

cpmbailey commented 1 year ago

Yes, think I used the master branch as of this morning

JeffyCN commented 1 year ago

I still get some timeout getting decoded frame messages but this seems to complete for me now.

timeout logs are for debugging only, since we are doing non-block(or 1ms) polling for async decoding.

JeffyCN commented 1 year ago

This play the video (does not work via SSH). Here I am using mppvideodec @JeffyCN how this can work if mpi_dec_test reports Segmentation fault and apparently does not work? There are many strange things here to understand

could be mpp test's bug or using wrong command options: mpi_dec_test -t 7 -i test.h264