Hardware scaling for decoder

[MarcA711]

The rkmpp_xxx_encoders allow to pass -width and -height to scale the frames using hardware acceleration.

Is it possible to use those options when only decoding? This would be useful for an application like frigate that only decodes and scales frames but does not transcode.

However, great project. Thank you really much!

[hbiyik]

    /* video only */
    /**
     * picture width / height.
     *
     * @note Those fields may not match the values of the last
     * AVFrame output by avcodec_receive_frame() due frame
     * reordering.
     *
     * - encoding: MUST be set by user.
     * - decoding: May be set by the user before opening the decoder if known e.g.
     *             from the container. Some decoders will require the dimensions
     *             to be set by the caller. During decoding, the decoder may
     *             overwrite those values as required while parsing the data.
     */
    int width, height;

i received similar question in the radxa forums, i think you need a specific input size to limit the parallel processing on the video output.

The thing is this decoder already abused too much in terms of ffmpeg conventions, i pasted above the ffmpeg api, it seems that it is not the prohibited for decoder to change the picture size. I already have a lot of RGA chains in the decoder, i can add one input to resize in one of them, but i have too see if this overcomplicates the code or not first.

[hbiyik]

And you wont be able to change the fps, only the dimentions

[MarcA711]

Thank you for your explanation. Does this mean that changing fps is impossible, even in software? Or that changing fps will not be hardware accelerated? Is it possible to change fps when transcoding using hardware acceleration? Is changing fps in software cpu inteniv?

[hbiyik]

i can only talk about the rkmppdecoder* i have and it wont support changing the fps in decoder (hardware does not supprot it), can may be supported in encoder but that wont help you.

I am sure there should be a way to this in ffmpeg, after the decoding is done, reducing the FPS should not be cpu intensive, can not say the same for the increasing FPS. yet i do not know, i might be developing a decoder for ffmpeg, still im still a noob as an ffmpeg user. you can ask someone who is expert..

[hbiyik]

added in https://github.com/hbiyik/FFmpeg/commit/ee436e35f0607914a23a7cecf2bb002bde50e0b9 please get the latest from exp_refactor_all branch.

decoder has now -width & -height arguments, i have tested with below commands they seem to be working.

mpv --hwdec=rkmpp --msg-level=ffmpeg=debug --vd-lavc-o=width=640,height=480 video.mp4
mpv --msg-level=ffmpeg=debug --vd-lavc-o=width=640,height=480 video.mp4
ffplay -c:v h264 -width 640 -height 480 video.mp4 
ffmpeg -c:v h264_rkmpp_decoder -width 640 -height 480 -i video.mp4 -an -sn -f null -

also have a look at here https://github.com/hbiyik/FFmpeg/issues/37

To change fps have a look at this filter, this should not be cpu intensive: https://ffmpeg.org/ffmpeg-filters.html#fps-1

[MarcA711]

Incredible! Thank you for adding this feature and doing it so fast.

Keep up the great work.

[hbiyik]

@MarcA711 it seems that scaling was already implemented, but i wasnt exposing the width height to users, therefore it was a low hanging fruit. keep me informed if it does not work, or it does work.

[great9]

guys, we just want to use hardware scaling in frigate on rk3588 :) thanks @hbiyik for the awesome work

[hbiyik]

Closing, i think it works

[MarcA711]

Sorry, but I didn't have time so far to recompile FFmpeg, since I am restructuring my code as well and compiling takes hours on my platform.

However, if I recompile it in the near future and test it, I will let you know. Once again, sorry for the late reply!