When/how to obtain imageFromCurrentFramebuffer (for use in a CVPixelBuffer)

[Cloov]

Hi,

I'm capturing video with a AVCaptureSession utility of my own, that modifies the sample buffer's pixel buffer. I then add it into the GPUImage chain with this CVPixelBuffer GPUImageOutput variant:

https://github.com/YuAo/YUGPUImageCVPixelBufferInput

This works fine. And I'm able to take my capture session video frames through Dlib/OpenCV, mark the camera image with some face-tracking debug drawing, then it goes through GPUImage and I apply an effect filter or two. I also write this out to a GPUImageView so the user can see a preview. So my chain is:

pixelBufferInput.addTarget(effectFilter1) effectFilter1.addTarget(effectFilter2) effectFilter2.addTarget(myGpuImageView)

And as my camera frames come in, I have OpenCV/Dlib modify them directly, locking and unlocking the buffer. Then, they are fed to GPUImage with the following, via YUGPUImageCVPixelBufferInput:

pixelBufferInput.processCVPixelBuffer(myCapturedPixelBuffer)

The GPUImageView that's in my view hierarchy seems to filter the captured image in a timely manner, which is great, but:

I don't know how, or exactly when, to obtain a pixel buffer from the filtered GPUImage, and from which target.

I've tried using the imageFromCurrentFramebuffer method on the GPUImageView, and also the second effect filter, but those seem to always be nil at the point I try to access them. I have a method to convert that CGImage/UIImage to a pixel buffer, and I was hoping to then place that back into my original sample buffer.

I've seen many discussions on imageFromCurrentFramebuffer being nil, but they all seem to be around single image capture.

Can you provide any assistance, possibly on when/how I can reliably obtain an image from the current GPUImageOutput?

[Cloov]

I'm now trying to use GPUImageRawDataOutput for this, but bytesPerRow is always 0 for me.

weak var output = rawDataOutput!
rawDataOutput?.newFrameAvailableBlock = {
   guard let output = output else { return }
   output.lockFramebufferForReading()
   let bytesPerRow = output.bytesPerRowInOutput()
   let dataLength = Int(bytesPerRow) * height
   let nsData = NSData(bytes: output.rawBytesForImage, length: dataLength)
   print("\(bytesPerRow), \(output.rawBytesForImage)")
   output.unlockFramebufferAfterReading()
}

Any idea what I'm doing wrong here?

[Cloov]

I realised eventually that if I query bytesPerRowInOutput before rawBytesForImage, then it's 0. If I first do something like:

let bytes = output.rawBytesForImage let bytesPerRow = output.bytesPerRowInOutput

Both values are valid.