FFmpegFrameGrabber FFmpegFrameRecorder may lead to off-heap memory leak

[ZhanPro]

When I use JavaCV's FFmpegFrameGrabber and FFmpegFrameRecorder in Java to perform audio format conversion, an off-heap memory leak occurred.

Here is my code : `@Slf4j public class JavaCVDecoder implements IDecoder {

FFmpegFrameGrabber grabber;

FFmpegFrameRecorder recorder;

FFmpegFrameRecorder videoRecorder;

private int inputAudioChannels;

private AudioCodec inputAudioCodec;

private int inputSampleRate;

private long duration;

private int outputAudioChannels;

private AudioCodec outputAudioCodec;

private int outputSampleRate;

private String inputFormat;

private String outputFormat;

public int imageWidth;

public double frameRate;

public int imageHeight;

public int totalFrames;

public double totalSeconds;

public int bitrate;

@Override
public double getFrameRate() {
    return frameRate;
}

@Override
public int getImageWidth() {
    return imageWidth;
}

@Override
public int getImageHeight() {
    return imageHeight;
}

public JavaCVDecoder(InputStream inputStream, OutputStream outputStream, int audioChannels) {
    this.grabber = new FFmpegFrameGrabber(inputStream);
    this.recorder = new FFmpegFrameRecorder(outputStream, audioChannels);
}

public JavaCVDecoder(String inputFileName, String outputFileName, int audioChannels) {
    this.grabber = new FFmpegFrameGrabber(inputFileName);
    this.recorder = new FFmpegFrameRecorder(outputFileName, audioChannels);
}

public JavaCVDecoder(FFmpegFrameGrabber grabber, FFmpegFrameRecorder recorder) {
    this.grabber = grabber;
    this.recorder = recorder;
}

public JavaCVDecoder(ByteArrayInputStream inputStream, ByteArrayOutputStream audioOutput,
                     ByteArrayOutputStream videoOutput) {
    this.grabber = new FFmpegFrameGrabber(inputStream);
    this.recorder = new FFmpegFrameRecorder(audioOutput, 1);
    this.videoRecorder = new FFmpegFrameRecorder(videoOutput, this.grabber.getImageWidth(),
            this.grabber.getImageHeight());
}

@Override
public void start() {
    try {
        this.grabber.start();
        this.recorder.start();

        int inputCodecId = this.grabber.getAudioCodec();
        this.inputAudioCodec = this.getAudioCodecByCodecId(inputCodecId);

        this.inputAudioChannels = this.grabber.getAudioChannels();
        this.inputSampleRate = this.grabber.getSampleRate();
        this.duration = this.grabber.getLengthInTime() / Constant.MICROSECONDS_PER_MILLISECOND;
        this.inputFormat = this.grabber.getFormat();

        int outputCodecId = this.recorder.getAudioCodec();
        this.outputAudioCodec = this.getAudioCodecByCodecId(outputCodecId);
        this.outputAudioChannels = this.recorder.getAudioChannels();
        this.outputSampleRate = this.recorder.getSampleRate();
        this.outputFormat = this.recorder.getFormat();

        Frame frame;
        while ((frame = this.grabber.grabFrame()) != null) {
            if (frame.type == Frame.Type.AUDIO) {
                this.recorder.record(frame);
                frame.close();
            }
        }
    } catch (FFmpegFrameGrabber.Exception e) {
        log.error("ffmpeg frame grabber start exception:{}", e.getMessage(), e);
        throw new RuntimeException(e);
    } catch (FFmpegFrameRecorder.Exception e) {
        log.error("ffmpeg frame recorder start exception:{}", e.getMessage(), e);
        throw new RuntimeException(e);
    } catch (FrameGrabber.Exception e) {
        log.error("frame grabber exception:{}", e.getMessage(), e);
        throw new RuntimeException(e);
    }
}

`

[ZhanPro]

The version is 1.5.8

org.bytedeco javacv-platform 1.5.8

[saudet]

Please try again with the snapshots: http://bytedeco.org/builds/

[ZhanPro]

Oh, thanks, I‘ll try it

[ZhanPro]

Please try again with the snapshots: http://bytedeco.org/builds/ So it's a known issue in 1.5.8 ?

[saudet]

Duplicate of #2255

[ZhanPro]

I have followed the instructions to upgrade the version of JavaCV to 1.5.11-SNAPSHOT. However, I encountered an issue: after the upgrade, my program runs fine on macOS when packaged into a JAR, but it fails to run on Linux, with the following error:

Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.bytedeco.ffmpeg.global.avutil at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:348) at org.bytedeco.javacpp.Loader.load(Loader.java:1289) at org.bytedeco.javacpp.Loader.load(Loader.java:1234) at org.bytedeco.javacpp.Loader.load(Loader.java:1210) at org.bytedeco.ffmpeg.avformat.Read_packet_Pointer_BytePointer_int.(Read_packet_Pointer_BytePointer_int.java:45) at org.bytedeco.javacv.FFmpegFrameGrabber.(FFmpegFrameGrabber.java:372) at com.sankuai.ai.speech.platform.codec.decoder.javacvdecoder.JavaCVDecoder.(JavaCVDecoder.java:84) at com.sankuai.ai.speech.platform.codec.uitls.DecodeUtils.decode(DecodeUtils.java:98) at com.sankuai.ai.speech.platform.api.gateway.component.AudioNormalizeComponent.process(AudioNormalizeComponent.java:79)

The dependencies I am using are:

`

org.bytedeco

            <artifactId>javacv</artifactId>
            <version>1.5.11-SNAPSHOT</version>
        </dependency>

        <dependency>
            <groupId>org.bytedeco</groupId>
            <artifactId>javacpp</artifactId>
            <version>1.5.11-SNAPSHOT</version>
        </dependency>

        <dependency>
            <groupId>org.bytedeco</groupId>
            <artifactId>ffmpeg-platform</artifactId>
            <version>7.0-1.5.11-SNAPSHOT</version>
        </dependency>

`

What is the reason for this?

[ZhanPro]

操作系统 CentOS 7.6.1810 内核版本 Linux 5.10.0-136.16.0.mt20230627.508.mt2203sp1.x86_64 CPU 型号 Intel(R) Xeon(R) CPU E5-2630 v4 @ 2.20GHz

[ZhanPro]

I sincerely look forward to your reply.

[saudet]

Please try again with Ubuntu

On Mon, Jul 29, 2024, 17:15 ZhanPro @.***> wrote:

I sincerely look forward to your reply.

— Reply to this email directly, view it on GitHub https://github.com/bytedeco/javacv/issues/2256#issuecomment-2255292085, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAZMQFYWVWMI5ASYXS6FJB3ZOX22BAVCNFSM6AAAAABLPV2KGWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENJVGI4TEMBYGU . You are receiving this because you modified the open/close state.Message ID: @.***>

[ZhanPro]

Please try again with Ubuntu … On Mon, Jul 29, 2024, 17:15 ZhanPro @.> wrote: I sincerely look forward to your reply. — Reply to this email directly, view it on GitHub <#2256 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAZMQFYWVWMI5ASYXS6FJB3ZOX22BAVCNFSM6AAAAABLPV2KGWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENJVGI4TEMBYGU . You are receiving this because you modified the open/close state.Message ID: @.>

However, all our online servers are running on CentOS.

[ZhanPro]

so, is there any other way?