Looking to contribute

[JohnCoates]

Hey Brad,

I'm looking to contribute code to this project. I don't have any OpenGL experience, but I've been coding Swift since day one.

What are you looking for as far as contributions? Would you be interested in having any code cleaned up? I see that GaussianBlur.swift has shaders defined inline instead of their own separate files.

Would you be interested in a pull request that for example replaced the following function:

func vertexShaderForOptimizedGaussianBlurOfRadius(_ radius:UInt, sigma:Double) -> String {
    guard (radius > 0) else { return OneInputVertexShader }

    let optimizedOffsets = optimizedGaussianOffsetsForRadius(radius, sigma:sigma)
    let numberOfOptimizedOffsets = optimizedOffsets.count

    // Header
    var shaderString = "attribute vec4 position;\n attribute vec4 inputTextureCoordinate;\n \n uniform float texelWidth;\n uniform float texelHeight;\n \n varying vec2 blurCoordinates[\((1 + (numberOfOptimizedOffsets * 2)))];\n \n void main()\n {\n gl_Position = position;\n \n vec2 singleStepOffset = vec2(texelWidth, texelHeight);\n"
    shaderString += "blurCoordinates[0] = inputTextureCoordinate.xy;\n"
    for currentOptimizedOffset in 0..<numberOfOptimizedOffsets {
        shaderString += "blurCoordinates[\(((currentOptimizedOffset * 2) + 1))] = inputTextureCoordinate.xy + singleStepOffset * \(optimizedOffsets[currentOptimizedOffset]);\n"
        shaderString += "blurCoordinates[\(((currentOptimizedOffset * 2) + 2))] = inputTextureCoordinate.xy - singleStepOffset * \(optimizedOffsets[currentOptimizedOffset]);\n"
    }

    shaderString += "}\n"
    return shaderString
}

with this function:

func vertexShaderForOptimizedGaussianBlurOfRadius(_ radius:UInt, sigma:Double) -> String {
    guard (radius > 0) else { return OneInputVertexShader }
    let optimizedOffsets = optimizedGaussianOffsetsForRadius(radius, sigma:sigma)
    let numberOfOptimizedOffsets = optimizedOffsets.count
    let blurCoordinatesCount = 1 + (numberOfOptimizedOffsets * 2)
    let defines: [String: CustomStringConvertible] = [
        "BLUR_COORDINATES_COUNT": blurCoordinatesCount,
        "OPTIMIZED_OFFSETS_COUNT": numberOfOptimizedOffsets
    ]
    let shaderContents = OptimizedGaussianBlurVertexShader
    let preprocessedShader = preprocessShader(shader: shaderContents,
                                           defines: defines)
    return preprocessedShader
}

and puts the shader code in OptimizedGaussianBlur.vsh

[BradLarson]

Unfortunately, I haven't had much time to work on this project over the last several months, outside of my immediate needs, as evidenced by the pull requests awaiting my review.

I don't really see much of an advantage in the reorganization you propose. I have to programmatically generate the Gaussian blur fragment shaders, which require a bit more than simple replacement, and having the vertex shaders be generated in the same way makes the code consistent. It also allows for future modifications where I might want to rewrite the vertex shaders in a more elaborate way. It's probably not worth the effort to restructure this.

These are the areas that are currently missing from GPUImage2 that are present in the original project:

• a GPUImageUIElement equivalent
• a Hough transform parallel coordinate line detector (pull request awaiting review for this)
• still photo camera capture, as an extension of the video camera
• synchronization of movie writing when re-encoding movies from disk, for faster-than-realtime reencoding
• audio re-encoding from movies

I'm currently working to resolve a compilation issue around using the Video4Linux library on Linux systems, which is the last issue before I restore full Linux compatibility for Swift 3. If you uncomment the code in V4LCamera.swift and try to build that on Linux, you'll see what I mean.

Beyond these, I've identified these areas to work on:

• Raw data inputs and outputs need to support texture caches on iOS
• Still image outputs need texture cache support
• Filter groups need to support masking in the same way that standalone filters do
• Texture cache support needs to be added to the Mac side of things for 10.11+ systems
• Add an optimization that detects when filters only need luminance in the chain, cut out grayscale and RGB conversions and just pass along luminance part of camera input
• Split RGB input into YUV planes for accelerating movie encoding

None of these are particularly easy tasks, and will require some understanding of how OpenGL rendering occurs within the framework, but these are the major items that are lacking right now.

[JohnCoates]

I totally get not having time to review pull requests. I'm the same way with my open source projects, so I'm thankful you took the time to write out this thorough reply.

I get why you don't see an advantage in the partial restructuring, and I better understand the shader requirements now. I actually spent the day converting the fragment shader to its own .fsh file, before stumbling across Apple's Best Practices for Shaders page and realizing that you optimized the shader to eliminate loops and reduce branching, and that having the code in a .fsh file would mean eliminating these very useful optimizations.

I'll review your list and start off with something small to learn my way around OpenGL and your framework.

[gaming-hacker]

i'd like to see if you can use some of the new metal framework in swift.

[BradLarson]

@gaming-hacker - See my response here: https://github.com/BradLarson/GPUImage2/issues/5#issuecomment-211504303