bytedeco / javacv

Java interface to OpenCV, FFmpeg, and more
Other
7.57k stars 1.58k forks source link

4 channel frame will cause crash in libswscale.so #1914

Closed ggslayer closed 2 years ago

ggslayer commented 2 years ago

Hi, in my sence, input video file's pixel fomrat has alpha channel, I use grabber grab frame, do some operate on the frame's image, then convert to frame again , put it in the recorder, then crash. please give some advise for the crash, thank you! code:

`

    FFmpegLogCallback.set();

    frameGrabber = new FFmpegFrameGrabber(srcfile);
    frameGrabber.setPixelFormat(avutil.AV_PIX_FMT_RGBA);

    Frame captured_frame = null;

    try {
        frameGrabber.start();

        recorder = new FFmpegFrameRecorder(destfile, frameGrabber.getImageWidth(), frameGrabber.getImageHeight(), frameGrabber.getAudioChannels());
        recorder.setInterleaved(true);
        recorder.setVideoOption("tune","zerolatency");
        recorder.setVideoOption("preset", "ultrafast");
        recorder.setVideoCodec(avcodec.AV_CODEC_ID_H264);
        recorder.setFormat("flv");
        //recorder.setPixelFormat(avutil.AV_PIX_FMT_RGBA);   // this is not supported.
        recorder.setFrameRate(frameGrabber.getFrameRate());
        recorder.setVideoBitrate(frameGrabber.getVideoBitrate());

        recorder.setAudioBitrate(192000);
        recorder.setAudioOptions(frameGrabber.getAudioOptions());
        recorder.setAudioQuality(0);
        recorder.setSampleRate(44100);
        recorder.setAudioCodec(avcodec.AV_CODEC_ID_AAC);
        recorder.start();

        frameGrabber.setTimestamp(0);

        while (true) {
            try {
                captured_frame = frameGrabber.grabFrame();

                if (captured_frame == null) {
                    System.out.println("!!! Failed cvQueryFrame");
                    break;
                }

                if(captured_frame.type == Frame.Type.VIDEO){
                    captured_frame = MixFrame(captured_frame);  // do addweight with other png with 4 channels
                    Thread.sleep(20);
                }else{
                    //System.out.println("audio frame");
                }

                recorder.record(captured_frame);            // Crash here!!!
            } catch (Exception e) {
                e.printStackTrace();
            }
        }
    }

public static Frame MixFrame(Frame frame){
    Mat frame_image = matConv.convertToMat(frame);
    Mat result = imgBg.clone();

    imageBlend(result, frame_image, 0, 0, 100, 400);
    Frame new_frame = matConv.convert(result);

    result.release();
    frame_image.release();
    return new_frame;
}

 public static void imageBlend(Mat bottom, Mat top, double bottom_alpha, double top_aplha, int top_pos_x, int top_pox_y){
    Mat ROI = bottom.apply(new Rect(top_pos_x, top_pox_y, top.arrayWidth(), top.arrayHeight()));
    opencv_core.addWeighted(ROI, bottom_alpha, top, top_aplha, 0.0, ROI);
    ROI.release();
}

`

saudet commented 2 years ago

Did you instead try to call recorder.record(captured_frame, AV_PIX_FMT_RGBA)?

ggslayer commented 2 years ago

thank you! It works