RenderView blocks thread when containing view's 'wantsLayer' property is set to 'true' [macOS]

[rorz]

The following applies to macOS (may not for iOS)

Thanks for the rapid addition of PictureInput to the repo. Found this while pipelining it into a RenderView and wanted to share as it may help fellow experimenters of GPUImage3.

TL;DR

Containing a RenderView in an NSView whose wantsLayer property is set to true causes thread blocking after a few, continuous/rapid pipeline-passes.^ This seems to be default behaviour in InterfaceBuilder that you can fix in affected XIBs' XML content, or by setting-up your views programmatically. ^ via .processImage()

Context

I'd been stumped for a while on a simple personal project with a pipeline as follows: PictureInput --> Filter --> RenderView Whereby the filter's value is controlled by a continuous NSSlider. After a few render-passes (via PictureInput.processImage(), the UI completely froze up and would no longer render the filter. The application used all of my CPU, and profiling it in Instruments showed the main Thread was being blocked:

After trying to replicate the SimpleImageFilter example project and still getting the issue, I found a difference between the example project's MainMenu.xib's XML content and mine — whereby the containing view (in my app) was set as follows: <view key="contentView" wantsLayer="YES" id="xxx-xx-xxx">

I didn't realise this was something you could set in IB, and I don't think you can, so I got rid of the property on that XML line and it now performs as well as the example project.

This was a vanilla project on Xcode 10 (beta 3) — so I'm unsure if it applies to 9 or below. This wasn't something I experienced an issue with on the OpenGL-based RenderView (in GPUImage2).

I know it's not a wise idea to mess around with layers on the render view directly, but I don't know why this behaviour happens as a result of a containing view's layer status?

[rorz]

Update:

Ran into a similar issue with an existing project (using GPUImage) that uses CALayer manipulation elsewhere in the ViewController. Not sure if it's directly related, but was getting a few [CAMetalLayerDrawable present] should not be called after already presenting this drawable. Get a nextDrawable instead. errors in the debugger before it eventually blocks the thread again.

This is from rapid-fire continuous processImage() calls (from a custom control), which I assume is outdoing the GPU by sending new textures too quickly.

Somewhat fixed it by adding a semaphore to the RenderView's draw call:

public override func draw(_ rect:CGRect) {
    _ = inFlightSemaphore.wait(timeout: DispatchTime.distantFuture)

    if let commandBuffer = sharedMetalRenderingDevice.commandQueue.makeCommandBuffer() {
      let semaphore = inFlightSemaphore
      commandBuffer.addCompletedHandler { (_ commandBuffer)-> Swift.Void in
        semaphore.signal()
      }

      if let currentDrawable = self.currentDrawable,
        let imageTexture = currentTexture {

        let outputTexture = Texture(orientation: .portrait, texture: currentDrawable.texture)

        commandBuffer.renderQuad(pipelineState: renderPipelineState, inputTextures: [0:imageTexture], outputTexture: outputTexture)

        commandBuffer.present(currentDrawable)

      }

      commandBuffer.commit()

    }
  }

Based off this SO answer and the boilerplate from the iOS Metal Game project starter.

Still manages to block eventually, so I'll look at less CPU-intensive ways of firing draw calls while testing it out for now :)