Is it possible to render via callback if an input image is required?

[mblaughton]

First off, this library is so clever and organized! It has to be said -- thanks to everyone involved. I’m more and more impressed the deeper I dive in.

I’m porting an algorithm from C++/OpenCV to Swift/GPUImage and would really appreciate it if a GPUImage veteran could tell me if my approach makes sense here. I haven't finished it yet, but even if I get it working I'll be a little worried about the potential race condition described below so I wanted to ask if it's a non-issue.

I have a relatively expensive operation that I’d like to perform on each of the corner candidates identified by HarrisCornerDetector. My structure is similar to the HarrisCornerDetector example in the FilterShowcase app, except my expensive operation would be used where the CrosshairGenerator operation is shown below, but unlike CrosshairGenerator it also takes an input (myFilter).

Example modified to show my situation:

let blendFilter = AlphaBlend()
let myFilter = SomeRandomFilter()
let crosshairGenerator = CrosshairGenerator() // Like official one, but modified to take 1 input
let harrisCornerDetector = HarrisCornerDetector()

harrisCornerDetector.cornersDetectedCallback = { corners in
   crosshairGenerator.renderCrosshairs(corners)
}

camera --> blendFilter --> outputView

camera --> myFilter --> crosshairGenerator
camera --> harrisCornerDetector // --> crosshairGenerator (via callback/almost like a pipe)

crosshairGenerator --> blendFilter

My question is this:

harrisCornerDetector provides corners to crosshairGenerator so it can render an output image. But is it possible that crosshairGenerator.renderCrosshairs() might execute before it finishes receiving the required input from myFilter? Or, does the pipeline approach in GPUImage prevent this from happening?

[BradLarson]

The crosshairGenerator can only take in a single input, so you won't be able to have it both take in the results of your filter and those of the Harris corner detector.

In general, filters proceed in the sequence they are specified, and are deterministic in their order of execution. You wouldn't need to worry about a race condition, but again the crosshair generator is only set up to take a single filter as input.

[cuhte3]

@mblaughton could you please share your modified generator which as I understood takes texture for crosshair as input? im new to gpuiimage and that exactly what I'm trying to achieve. thanks

[mblaughton]

@cuhte3 in my case I found that the fastest approach was to feed 2 textures into the "expensive" operation, where:

1. Texture 1 is the go/no-go mask stating whether the expensive calculation should happen
2. Texture 2 holds the inputs for the expensive calculation

Then the shader for the expensive operation has a simple "if()" early on that performs a go/no-go test against Texture 1 and returns immediately if the answer is no-go.

I was concerned about using a branching if() in the shader, but in my case it ended up being really fast and avoided an additional read-back from the GPU.

Ultimately I didn't even need to use this approach because I was able to do almost everything with regular TextureSamplingOperation and a handful of custom shaders. It just took me a while to understand the library well enough to understand how simple it could be.

Update: I thought I had already thanked Brad for his helpful answer above but apparently I didn't. Thanks @BradLarson!