Grabber is at a risk of crashing

[CJL6015]

There is a risk of crash when the grabber restarts and closes, if the pixel format is set to BGRA grabber.setPixelFormat(avutil.AV_PIX_FMT_BGRA)

[saudet]

Why? What's special about that pixel format?

[CJL6015]

I don't know the exact reason either. After many tests, I found that as long as this pixel format is set, the program will crash when the close method is called.

[saudet]

Could you provide a small self-contained example to reproduce this issue?

[davepiar]

I also found the same problem when using the AV_PIX_FMT_RGBA. Seems any 32 bit format would cause a JVM crash. The sample code I used to reproduce it is:

public class DemoAVFreeCrash {

  public static void main(String[] args) throws Exception {

    IntStream.range(0, 10).forEach(i -> {
      System.out.println("Starting grabber: times=" + i);

      try (FFmpegFrameGrabber grabber = new FFmpegFrameGrabber(new FileInputStream(new File("C:\\Users\\user\\Desktop\\video.webm")))) {
        grabber.setPixelFormat(avutil.AV_PIX_FMT_RGBA);
        grabber.start();

        // Grab a Frame and properly close it
        try (Frame frame = grabber.grabImage()) {
          System.out.println("Frame grabbed: " + frame.timestamp);
        }
      } catch (Exception e) {
        e.printStackTrace();
      } finally {
        System.out.println("Finnaly reached!");
      }

      System.out.println("Closed grabber successfully: times=" + i);
    });
  }
}

And the output is:

Starting grabber: times=0
Frame grabbed: 0
Finnaly reached!
Closed grabber successfully: times=0
Starting grabber: times=1
Frame grabbed: 0

It never completes more than 2-3 iterations. Always is failing when freeing the frame av_free(image_ptr[i]); I've reproduced the same behavior in the latest javacv 1.5.9. I've also tried different video formats (mp4/webm) and always is crashing at closing the resource.

Do you have any idea on why the pixel format with 32 bit could produce this issue?

Thanks in advance for your awesome library!

[saudet]

Please try again with the snapshots: http://bytedeco.org/builds/

[davepiar]

I've run the same sample code with current 1.5.10-SNAPSHOT.

Unfortunately the result was even worse, now I never can finish one iteration. I always get this output:

Starting grabber: times=0
Frame grabbed: 0

Again, the issue only appears when using 32 bit pixel format (AV_PIX_FMT_RGBA, AV_PIX_FMT_BGRA, ...)

[saudet]

I'm not able to reproduce this here. The code above works fine for me with 1.5.10-SNAPSHOT

[davepiar]

Seems the issue is also very related to the video dimensions. I can reproduce it with the following video of 422x194: https://github.com/davepiar/sample-videos/blob/main/sample.webm

Maybe it is related to a memory alignment issue or something.

[saudet]

It's possible. There used to be a bug in FFmpeg requiring this hack: https://github.com/bytedeco/javacv/blob/1.5.9/src/main/java/org/bytedeco/javacv/FFmpegFrameGrabber.java#L1144 Could you try to remove those lines and see if it fixes this issue for you?

[davepiar]

Yes, I've already checked that issue. Removing this does not fix it, seems this workaround is still necessary but it's only working when using 24 bit pixel formats like RGB24. I found a new workaround setting the linesize alignment to 64 instead of 32. I've stress tested all a bunch of videos with pixel format RGBA and 64 alignment and now is working as expected.

This solution is not nice because of a waste of memory, but it works for me. Thanks @saudet for your support!

[saudet]

Sounds like a good enough workaround though. Please open a pull request!

[CJL6015]

Yes, I've already checked that issue. Removing this does not fix it, seems this workaround is still necessary but it's only working when using 24 bit pixel formats like RGB24. I found a new workaround setting the linesize alignment to 64 instead of 32. I've stress tested all a bunch of videos with pixel format RGBA and 64 alignment and now is working as expected.

This solution is not nice because of a waste of memory, but it works for me. Thanks @saudet for your support!

Is setting grabber.setPixelFormat(AV_PIX_FMT_BGRA64LE) the solution to this problem?

[saudet]

@CJL6015 No, I've pushed @davepiar's fix. Please give it a try with the snapshots: http://bytedeco.org/builds/

[CJL6015]

@saudet Thank you for your reply

[saudet]

Fix released with version 1.5.10. Thanks for reporting!