Encoder endofstream problem (lost few final frames)

[mcerveny]

Hello. I found the problem in encoder logic. 1) if packet (encoded frame) is not available (still encoding), EAGAIN is returned https://github.com/nyanmisaka/ffmpeg-rockchip/blob/27136b05e0742450a019d6bab80d840ac5fd6421/libavcodec/rkmppenc.c#L687-L691 2) error return is handled in encode function, *got_packet=0 https://github.com/nyanmisaka/ffmpeg-rockchip/blob/27136b05e0742450a019d6bab80d840ac5fd6421/libavcodec/rkmppenc.c#L776-L779 3) encoder loop check got_packet and if in draining state, ends draining and lost all remaining encoded packet https://github.com/nyanmisaka/ffmpeg-rockchip/blob/27136b05e0742450a019d6bab80d840ac5fd6421/libavcodec/encode.c#L353-L354

I did not discover why "ffmpeg" loop does not suffer this problem but my program have this problem. So I try to add busy-loop over rkmpp_get_packet() (if loop is only to "get:" label it sometimes stops for few seconds, please review the logic):

diff --git a/libavcodec/rkmppenc.c b/libavcodec/rkmppenc.c
index 1688acc..afbb59f 100644
--- a/libavcodec/rkmppenc.c
+++ b/libavcodec/rkmppenc.c
@@ -774,6 +774,8 @@ send:

 get:
     ret = rkmpp_get_packet(avctx, packet);
+    if (!frame && ret == AVERROR(EAGAIN))
+        goto send;
     if (ret == AVERROR_EOF ||
         ret == AVERROR(EAGAIN))
         *got_packet = 0;

Typical problem (penultimate column shows PTS inserted to encoder, last column shows PTS outputted from encoder, so lost 2 frames (1456376400, 1456380000)):

main.c:712: flt->enc                     1456369200
main.c:709: >ENC
main.c:720: .ENC
main.c:744: enc->out                               1456358400
main.c:720: .ENC
main.c:744: enc->out                               1456362000
main.c:720: .ENC
main.c:709: >ENC
main.c:712: flt->enc                     1456372800
main.c:709: >ENC
main.c:712: flt->enc                     1456376400
main.c:709: >ENC
main.c:720: .ENC
main.c:744: enc->out                               1456365600
main.c:720: .ENC
main.c:709: >ENC
main.c:712: flt->enc                     1456380000
main.c:698: =send EOF to ENC (signaling EOF to encoder)
main.c:720: .ENC
main.c:744: enc->out                               1456369200
main.c:720: .ENC
[h264_rkmpp @ 0x557c23e3b0] End of stream
[h264_rkmpp @ 0x557c23e3b0] Wrote 0 bytes to encoder
[h264_rkmpp @ 0x557c23e3b0] Received a packet
main.c:744: enc->out                               1456372800
main.c:720: .ENC
[h264_rkmpp @ 0x557c23e3b0] End of stream
[h264_rkmpp @ 0x557c23e3b0] Wrote 0 bytes to encoder
[h264_rkmpp @ 0x557c23e3b0] Failed to get packet from encoder output queue: -11
main.c:733: =send EOF to OUT (received EOF from encoder)
======================= aframe 100 98

[nyanmisaka]

@mcerveny I managed to find a ffmpeg command that can be used to reproduce the problem. And your patch did fix it.

It seems that ffmpeg is not expecting codec->encode() to return EAGAIN while the encoder is draining/at eof (!frame). So what you added makes sense.

ffmpeg -init_hw_device rkmpp \
-f lavfi -i color=c=cyan:s=1920x1080:d=4:r=25,format=nv12 \
-f lavfi -i testsrc2=s=960x540:d=4:r=25,format=bgra \
-lavfi "[0:v]hwupload[main];[1:v]hwupload[over];[main][over]overlay_rkrga=format=nv12:afbc=1:eof_action=repeat:repeatlast=1,scale_rkrga=w=1280:h=720:afbc=1" \
-c:v h264_rkmpp -v debug -y /tmp/100frames.mp4

[nyanmisaka]

Closed by 7a0200b