BradLarson / GPUImage2

GPUImage 2 is a BSD-licensed Swift framework for GPU-accelerated video and image processing.
BSD 3-Clause "New" or "Revised" License
4.85k stars 605 forks source link

Can I filter a image while showing live camera? #277

Closed Topwiz closed 5 years ago

Topwiz commented 5 years ago

I added a code like this to use capturePhoto on GPUImage2, Camera.swift file.

public func captureStillImage(delegate: AVCapturePhotoCaptureDelegate) { guard let photoOutPut = self.photoOutput else { return } let settings = AVCapturePhotoSettings() let previewFormat = [kCVPixelBufferPixelFormatTypeKey as String: Int(kCVPixelFormatType_32BGRA)] settings.previewPhotoFormat = previewFormat settings.isAutoStillImageStabilizationEnabled = true photoOutPut.capturePhoto(with: settings, delegate: delegate) }

And in the MainViewController when I call the capturePhoto, I'm able to get the CMSampleBuffer of the Image from "AVCaptureVideoDataOutputSampleBufferDelegate".

The problem is, the CMSampleBuffer will be always the Original Image not the filtered Image. so when I get the "didFinishProcessingPhoto" I need to filter the original image again before saving to the library. For filtering still image, I need to use this code.

let testImage = UIImage(named:"WID-small.jpg")! let toonFilter = SmoothToonFilter() let filteredImage = testImage.filterWithOperation(toonFilter)

or

let testImage = UIImage(named:"WID-small.jpg")! let toonFilter = SmoothToonFilter() let luminanceFilter = Luminance() let filteredImage = testImage.filterWithPipeline{input, output in input --> toonFilter --> luminanceFilter --> output }

or

let toonFilter = SmoothToonFilter() let testImage = UIImage(named:"WID-small.jpg")! let pictureInput = PictureInput(image:testImage) let pictureOutput = PictureOutput() pictureOutput.imageAvailableCallback = {image in // Do something with image } pictureInput --> toonFilter --> pictureOutput pictureInput.processImage(synchronously:true)

But, when I use this while showing live camera, it gets an error on pictureOutput that gets nil. I fined the problem for this it is because of I am using the "filter" on live camera so I think that can't filter with that. So I made a same filter like "toonFilter1" and changed this code "pictureInput --> toonFilter --> pictureOutput" to pictureInput --> toonFilter1 --> pictureOutput

And I got a filtered image to save in the library but! here is my real problem.

When I am showing the live camera with filter, and converting UIImage again before saving image. the camera gets a lot of slowing for about 2,3 sec. and getting a little blink on live camera. and the cpu goes up to about 30%. and gets an Warning message not always but some time, "Warning: tried to add target beyond target's input capacity --> Pipeline.swift: addTarget(_:atTargetIndex:): 43" I think it saves with a filtered image but it isn't a good way to do this.

Is there a different way to save filtered image while showing live filter image?

Im using this because I can't get 3024 / 4032 picture for just capturing videoOuptPut.(preset.Photo) it gets 750 / 1000