FFmpegFrameGrabber.grab() occasionally skipped when extract all frames from video with HW codec in android

[tjsm-dev]

[Environment] JavaCV: 1.5.7 OS: Android 30 Device: LG V50(LM-V500N)

[What i'm trying to do]

• Extract all frames from MP4 video
• Run on background thread

[Description of the problem situation] My program works fine when I use software codecs. I got the same result N times(93 frames were extracted from the reference video).

When the program is executed after writing the code to use the hardware codec, the first execution is extracted with the same number of frames as when using the software codec.

But FFmpegFrameGrabber.grab() occasionally skipped(returned null) sometimes. As a result, the total number of frames is different each time. 1st: 93 frames 2nd: 91 frames 3rd: 88 frames ...

[Video extraction code]

FFmpegFrameGrabber.tryLoad()
val bitmapConverter = AndroidFrameConverter()
val inputStream = FileInputStream(path)
val grabber = FFmpegFrameGrabber(inputStream)

try {
    FFmpegLogCallback.set()
    grabber.pixelFormat = AV_PIX_FMT_RGBA
    grabber.videoCodecName = "h264_mediacodec"
    grabber.setOption("fflags", "genpts")
    grabber.setVideoOption("threads", "0")
    grabber.start()

    var totalFrameCount = 0
    for (currFrameIndex in 0 until grabber.lengthInVideoFrames) {
        val frame = grabber.grabImage()
        val src = bitmapConverter.convert(frame) ?: continue
        totalFrameCount++
     }
    logD("DEBUG", "total frame count : $totalFrameCount")
} catch (e: Exception) {
    throw e
} finally {
    inputStream.close()
    grabber.release()
    bitmapConverter.close()
}

[Solutions I tried but failed]

1. Use startUnsafe() instead of start()
2. Use grabAtFrameRate() instead of grab() (https://github.com/bytedeco/javacv/commit/b29601401964831630b64ad0fba504a4dc8977e3)

Is there any other way I can try?

[saudet]

Don't call getLengthInVideoFrames(), it's an approximation that may not always be accurate.

[tjsm-dev]

Don't call getLengthInVideoFrames(), it's an approximation that may not always be accurate.

Thanks for your kind reply. Can you recommend an exact iteration method for extracting the whole frame?

[saudet]

grabImage() returns null at the end of the stream.

[saudet]

I see, you're saying that MediaCodec doesn't do that reliability. That's a problem with FFmpeg or Android, so we'll need to report that upstream, unless @tmm1 has an idea?

[saudet]

Also, please try with the snapshots since maybe that has already been fixed somewhere: http://bytedeco.org/builds/

[tjsm-dev]

Also, please try with the snapshots since maybe that has already been fixed somewhere: http://bytedeco.org/builds/

Thanks, I tried with the version you provided: '1.5.8-SNAPSHOT', but the result was the same as 1.5.7.