NativeDeallocator do not release memory problem

[linlinwen]

when I use FFmpegFrameGrabber to get screenshot or convert video. the previous tasks ware successful, but the following tasks will crash with OutOfMemoryError: Physical memory usage is too high.

here is the error log:

2023-09-04 14:59:12.802 ERROR 51748 --- [org.springframework.amqp.rabbit.RabbitListenerEndpointContainer#0-1] c.i.p.r.c.MediaProcessTaskConsumer       : Physical memory usage is too high: physicalBytes (1088M) > maxPhysicalBytes (1000M)

java.lang.OutOfMemoryError: Physical memory usage is too high: physicalBytes (1088M) > maxPhysicalBytes (1000M)
    at org.bytedeco.javacpp.Pointer.deallocator(Pointer.java:585) ~[javacpp-1.4.4.jar!/:1.4.4]
    at org.bytedeco.javacpp.Pointer.init(Pointer.java:125) ~[javacpp-1.4.4.jar!/:1.4.4]
    at org.bytedeco.javacpp.avcodec$AVPacket.allocate(Native Method) ~[ffmpeg-4.1-1.4.4.jar!/:1.4.4]
    at org.bytedeco.javacpp.avcodec$AVPacket.<init>(avcodec.java:1531) ~[ffmpeg-4.1-1.4.4.jar!/:1.4.4]
    at org.bytedeco.javacv.FFmpegFrameGrabber.startUnsafe(FFmpegFrameGrabber.java:714) ~[javacv-1.4.4.jar!/:1.4.4]
    at org.bytedeco.javacv.FFmpegFrameGrabber.start(FFmpegFrameGrabber.java:705) ~[javacv-1.4.4.jar!/:1.4.4]

[saudet]

Please make sure FFmpegFrameGrabber.stop() gets called.

[linlinwen]

here is our project dependencies：

<dependency>
            <groupId>org.bytedeco</groupId>
            <artifactId>javacv</artifactId>
            <version>1.5.4</version>
            <exclusions>
                <exclusion>
                    <groupId>org.bytedeco</groupId>
                    <artifactId>*</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>org.bytedeco</groupId>
            <artifactId>javacpp</artifactId>
            <version>1.5.4</version>
        </dependency>
        <dependency>
            <groupId>org.bytedeco</groupId>
            <artifactId>ffmpeg</artifactId>
            <version>4.3.1-1.5.4</version>
        </dependency>
        <dependency>
            <groupId>org.bytedeco</groupId>
            <artifactId>ffmpeg</artifactId>
            <version>4.3.1-1.5.4</version>
            <classifier>linux-x86_64</classifier>
        </dependency>
        <dependency>
            <groupId>org.bytedeco</groupId>
            <artifactId>openblas</artifactId>
            <version>0.3.10-1.5.4</version>
            <classifier>linux-x86_64</classifier>
        </dependency>
        <dependency>
            <groupId>org.bytedeco</groupId>
            <artifactId>javacpp</artifactId>
            <version>1.5.4</version>
            <classifier>linux-x86_64</classifier>
        </dependency>

and process code just like that：

try (FFmpegFrameGrabber frameGrabber = new FFmpegFrameGrabber(file)) {
            // get screenshot
            frameGrabber.start();
            // duration (second)
            long duration = frameGrabber.getLengthInTime() / avutil.AV_TIME_BASE;
            Frame frame = locatorFactory(frameGrabber);
            Java2DFrameConverter converter = new Java2DFrameConverter();
            BufferedImage bufferedImage = converter.getBufferedImage(frame);
            ByteArrayOutputStream screenshotOutputStream = new ByteArrayOutputStream();
            ImageIO.write(bufferedImage, "jpg", screenshotOutputStream);
            InputStream imgInput = new ByteArrayInputStream(screenshotOutputStream.toByteArray());
            frameGrabber.stop();
            // upload screenshot
            String screenshot = MaterialConstant.SCREEN_SHOT_IMG.concat("/").concat(fileName);
            minioClient.putObject(PutObjectArgs.builder()
                .bucket(MaterialConstant.SCREEN_SHOT_IMG)
                .object(fileName)
                .stream(imgInput, imgInput.available(), -1)
                .build()
            );
            // update data
            dataPersistence(mediaProcessTask.getKeyName() , screenshot);
            log.info("this video duration is {} s , the screenshot in img bucket {}", duration, screenshot);
        } catch (Exception e) {
            dataPersistence(mediaProcessTask.getKeyName() , "");
            log.error("get video screenshot error: {}", e.getMessage(), e);
            e.printStackTrace();
        } finally {
            if (file.exists()) {
                file.delete();
            }
        }

[linlinwen]

Please make sure FFmpegFrameGrabber.stop() gets called.

yeah, we use try catch with resources to make sure close the FFmpegFrameGrabber.

[linlinwen]

the Pointer log do invoke the release method:

Debug: Releasing org.bytedeco.javacpp.Pointer$NativeDeallocator[ownerAddress=0x7fb89d076a70,deallocatorAddress=0x7fb72e867510]
Debug: Releasing org.bytedeco.javacpp.Pointer$NativeDeallocator[ownerAddress=0x7fb89d40a4d0,deallocatorAddress=0x7fb72e867510]
Debug: Releasing org.bytedeco.javacpp.Pointer$NativeDeallocator[ownerAddress=0x7fb89e4d4290,deallocatorAddress=0x7fb72a366980]
Debug: Releasing org.bytedeco.javacpp.Pointer$NativeDeallocator[ownerAddress=0x7fb89d63fed0,deallocatorAddress=0x7fb72a366980]

but it seem is not effected.

[saudet]

You're using an old version, please try again with the latest release.

[linlinwen]

You're using an old version, please try again with the latest release.

I had used version 1.5.7 before, but also did not effected. If we set properties -Dorg.bytedeco.javacpp.maxPhysicalBytes=0G, the process will not crash.

[linlinwen]

You're using an old version, please try again with the latest release.

hi, saudet.

I have try with version 1.5.9, but also do not release the direct memory.

the previous task log show it has invoke the release method:

Debug: Releasing org.bytedeco.javacpp.Pointer$NativeDeallocator[ownerAddress=0x7fe298756150,deallocatorAddress=0x7fe25407daf0]
Debug: Collecting org.bytedeco.javacpp.Pointer$NativeDeallocator[ownerAddress=0x0,deallocatorAddress=0x0]
Debug: Releasing org.bytedeco.javacpp.Pointer$NativeDeallocator[ownerAddress=0x7fe2987561c0,deallocatorAddress=0x7fe25407daf0]
Debug: Collecting org.bytedeco.javacpp.Pointer$NativeDeallocator[ownerAddress=0x0,deallocatorAddress=0x0]
Debug: Releasing org.bytedeco.javacpp.Pointer$NativeDeallocator[ownerAddress=0x7fe2987560a0,deallocatorAddress=0x7fe20b5801d0]
Debug: Collecting org.bytedeco.javacpp.Pointer$NativeDeallocator[ownerAddress=0x0,deallocatorAddress=0x0]

but the following task will crashed:

java.lang.OutOfMemoryError: Cannot allocate new PointerPointer(8): totalBytes = 784, physicalBytes = 1045M
    at org.bytedeco.javacpp.PointerPointer.<init>(PointerPointer.java:151) ~[javacpp-1.5.9.jar!/:1.5.9]
    at org.bytedeco.javacv.FFmpegFrameGrabber.startUnsafe(FFmpegFrameGrabber.java:921) ~[javacv-1.5.9.jar!/:1.5.9]
    at org.bytedeco.javacv.FFmpegFrameGrabber.start(FFmpegFrameGrabber.java:903) ~[javacv-1.5.9.jar!/:1.5.9]
    at org.bytedeco.javacv.FFmpegFrameGrabber.start(FFmpegFrameGrabber.java:898) ~[javacv-1.5.9.jar!/:1.5.9]
    at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) ~[spring-context-5.3.21.jar!/:5.3.21]
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[na:na]
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[na:na]
    at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) ~[na:na]
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[na:na]
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[na:na]
    at java.base/java.lang.Thread.run(Thread.java:829) ~[na:na]
Caused by: java.lang.OutOfMemoryError: Physical memory usage is too high: physicalBytes (1045M) > maxPhysicalBytes (1024M)
    at org.bytedeco.javacpp.Pointer.deallocator(Pointer.java:741) ~[javacpp-1.5.9.jar!/:1.5.9]
    at org.bytedeco.javacpp.Pointer.init(Pointer.java:127) ~[javacpp-1.5.9.jar!/:1.5.9]
    at org.bytedeco.javacpp.PointerPointer.allocateArray(Native Method) ~[javacpp-1.5.9.jar!/:1.5.9]
    at org.bytedeco.javacpp.PointerPointer.<init>(PointerPointer.java:143) ~[javacpp-1.5.9.jar!/:1.5.9]
    ... 13 common frames omitted

[saudet]

If we set properties -Dorg.bytedeco.javacpp.maxPhysicalBytes=0G, the process will not crash.

If that works, then continue to do that. Do you have another issue, or can we close this?

[linlinwen]

If we set properties -Dorg.bytedeco.javacpp.maxPhysicalBytes=0G, the process will not crash.

If that works, then continue to do that. Do you have another issue, or can we close this?

yeah, you can close the issue. but if i want fo figure out the cause, how can i do?

[saudet]

Well, 1GB isn't a lot of memory, so try to give it say 16 GB and if it runs fine, then I don't think there's any issues.

[linlinwen]

Well, 1GB isn't a lot of memory, so try to give it say 16 GB and if it runs fine, then I don't think there's any issues.

I get it，thanks a lot！