How to draw transparent frames in video

[ggslayer]

Hi, in code below, input a background image and a video with alpha channel. I want to give every frame a backgound image. but, every frame's transparent zone is black color, not transparent i except. please give me some help about my purpose

in the code, 1/2/3 code part is set to ABGR.

I guess:

1. tell the grabber to convert the origin pixel format to ABGR.
2. tell the recorder the image in the frame's pixel format is ABGR.
3. tell the graphics obj, to create a ABGR format's canvas, and all future drawed image's format is ABGR.

Is my guess correct? please give me some help about my purpose, thank you.

`

FFmpegLogCallback.set();

frameGrabber = new FFmpegFrameGrabber(srcfile); frameGrabber.setPixelFormat(avutil.AV_PIX_FMT_ABGR); // 1 set to ABGR

Frame captured_frame = null;

try { frameGrabber.start();

  recorder = new FFmpegFrameRecorder(destfile, frameGrabber.getImageWidth(), frameGrabber.getImageHeight(), frameGrabber.getAudioChannels());
  recorder.setInterleaved(true);
  recorder.setVideoOption("tune","zerolatency");
  recorder.setVideoOption("preset", "ultrafast");
  recorder.setVideoCodec(avcodec.AV_CODEC_ID_H264);
  recorder.setFormat("flv");

  recorder.setFrameRate(frameGrabber.getFrameRate());
  recorder.setVideoBitrate(frameGrabber.getVideoBitrate());

  recorder.setAudioBitrate(192000);
  recorder.setAudioOptions(frameGrabber.getAudioOptions());
  recorder.setAudioQuality(0);
  recorder.setSampleRate(44100);
  recorder.setAudioCodec(avcodec.AV_CODEC_ID_AAC);
  recorder.start();

  frameGrabber.setTimestamp(0);

  while (true) {
      try {
          captured_frame = frameGrabber.grabFrame();

          if (captured_frame == null) {
              System.out.println("!!! Failed cvQueryFrame");
              break;
          }

          if(captured_frame.type == Frame.Type.VIDEO){
              captured_frame = MixFrameDraw(captured_frame); 
          }else{
              //System.out.println("audio frame");
          }

          recorder.record(captured_frame, AV_PIX_FMT_ABGR);            // 2 is set ABGR
      } catch (Exception e) {
          e.printStackTrace();
      }
  }

}

public static Frame MixFrameDraw(Frame frame){ BufferedImage imgFrame = frameConverter.getBufferedImage(frame);

  BufferedImage res = new BufferedImage(bufferImgBg.getWidth(), bufferImgBg.getHeight(), BufferedImage.TYPE_INT_ABGR);  // 3 set to ABGR

  Graphics graphics = res.getGraphics();
  graphics.drawImage(bufferImgBg, 0, 0, null);

  //  video frame is smaller than background image, draw it a x,y offset
  // I except that here will draw a tranparent image, because video has alpha channel, but the transparent zone is black color 
  graphics.drawImage(imgFrame, 100, 100, null);   

  return frameConverter.getFrame(res);

}

`

[saudet]

Sounds like you might want to use a filter: https://www.ffmpeg.org/ffmpeg-filters.html

[ggslayer]

Now, I'm trying the graphics draw's direction, Is this road can go? the image from frameConverter.getBufferedImage have any different than normal image load from png? can you give me some advise on this direction? thank you I'm have little fear about filter, It looks so complex...

[CJL6015]

You can convert the frame to the Mat of OpenCV and blend them.