[hevc_rkmpp] Zerolatency encoding

[Gredys]

I am using hevc_rkmpp encoder in C++ and trying to figure out how to enable zero-latency mode. I.e. send frame = receive packet without delays, buffers and etc.

What I am trying to do - send real-time captures of game frames over the LAN, display and save them for later usage. I was using other implementation of ffmpeg for rkmpp (Orange PI 5 Plus) and summary delay wasn't bigger than 1 frame, but it had issues with bit-rate control. Now, with your project, I can easily control bit-rate and quality, but summary delay is bigger than 3-4 frames, which is not real-time for me. (its hard to react in action games with such delay)

Current problem with encoder is that I always get AVPacket for AVFrame with 1 frame delay. I.e. sending and receiving data like this:

send frame     : 0 - 1 - 2 - 3 - 4 - ...
receive packet : X - 0 - 1 - 2 - 3 - ...

Code flow and set options are pretty basic: (I'll skip really basic ones, those are to test zero-latency mode)

 - gop_size = 0
 - max_b_frames = 0
 - refs = 0
 - flags |= AV_CODEC_FLAG_LOW_DELAY
 - flags2 |= AV_CODEC_FLAG2_FAST
 - preset = ultrafast
 - tune = zerolatency

 1. receive frame from internal queue
 2. use `sws_scale` to convert to correct format
 3. `av_rescale_q` to fix pts
 4. `avcodec_send_frame` to send frame to encoder
 5. right after that `avcodec_receive_packet` but there is no data for the first frame
 6. doing steps from 1 to 4 and calling `avcodec_receive_packet` for the second time gives packet for the first frame

Is there is any option to force encoder give exactly one packet per frame without buffering it on its side?

P.S. It may be another issue, but will ask in any way - hevc_rkmpp decoder always crashes after working for a few seconds if using it from C++ code. Is it intended behavior or I need to create another issue?

[nyanmisaka]

 - gop_size = 0 //< I-frame only
 - max_b_frames = 0 //< not supported by hw
 - refs = 0 //< not supported by mpp
 - flags |= AV_CODEC_FLAG_LOW_DELAY
 - flags2 |= AV_CODEC_FLAG2_FAST //< not supported
 - preset = ultrafast //< not supported by mpp
 - tune = zerolatency //< not supported

 1. receive frame from internal queue
 2. use `sws_scale` to convert to correct format //< if possible use scale_rkrga to blit in hw
 3. `av_rescale_q` to fix pts
 4. `avcodec_send_frame` to send frame to encoder
 5. right after that `avcodec_receive_packet` but there is no data for the first frame
 6. doing steps from 1 to 4 and calling `avcodec_receive_packet` for the second time gives packet for the first frame

There is no so-called zero-latency mode in MPP. Latency can only be reduced by disabling async encoding, but it's definitely not zero.

I've just push several commits to support setting sync mode with the -flags +low_delay option, which is equal to the avctx->flags & AV_CODEC_FLAG_LOW_DELAY. This mode achieves one-input-one-output at the expense of lower FPS.

P.S. It may be another issue, but will ask in any way - hevc_rkmpp decoder always crashes after working for a few seconds if using it from C++ code. Is it intended behavior or I need to create another issue?

One ticket one issue. The description is too vague.

[nyanmisaka]

Closing as the requested feat has been implemented.