Masking via GPU_image?

[ephemer]

In iOS I can set a mask (just an 8bit alpha texture) on a CALayer and then render that layer as a solid rect containing either a texture or colour and only the high-alpha regions of the mask will be rendered by the GPU onto the frame buffer.

From what I understand this work would normally be done by a fragment shader. Is there an easier way in SDL_gpu?

[grimfang4]

I do my masking with a shader, but an easier way would be nice.

For simple rects, you can use clipping instead. For binary masking, I would consider adding stencil buffer support in some form or providing a built-in masking shader. You might be able to achieve a masking effect with a blend mode, but that would probably necessitate an intermediate blit (refer to tests/blend-mode).

[ephemer]

Hi, thanks for your reply! I'll look into the options and if I can come up with anything generalisable enough to put into SDL_gpu I'll make a pull request.

[grimfang4]

The new simple-shader-demo has an example of a kind of masking shader.

[grimfang4]

I'm thinking a stencil interface would be nice and then it's added functionality rather than just making something specifically easier. It'd be very useful with shaders, too. I'll consider this further.

[ephemer]

Sounds good!

[ephemer]

I just saw the bit about the simple-shader-demo, looks great and it's exactly what I needed to get started. Would be nice to have a kind of stencil interface, but the demo puts me in the right path for now. Not sure how I missed this back in July :) Still working on the project though in any case, thanks again for SDL_gpu

[ephemer]

Hi @grimfang4, I finally got to trying out the masking shaders. Forgive me if this is a silly question, but how do I get them working with shape rendering?

My mask is the same size as my intended output, so I've boiled the fragment shader down to just this:

in vec4 inColor;
in vec2 texCoord;
out vec4 fragColor;

uniform sampler2D mask_texture;

void main(void)
{
    vec4 maskColor = texture(mask_texture, texCoord);
    fragColor = vec4(inColor.rgb, inColor.a * maskColor.a);
}

What I'd like to do is something like this:

GPU_ActivateShaderProgram(mask_shader, &mask_block);
GPU_RectangleFilled(target, rect, sdlColor);
GPU_ActivateShaderProgram(0, NULL);

And have only the masked regions show up in the chosen color. The code as it's written doesn't appear to provide any input to the shader though. The maskColor is always (0.0, 0.0, 0.0, 0.0). Edit: seemingly because texCoord is always (0.0, 0.0). inColor does appear to provide the solid colour as provided by RectangleFilled

Do you have an idea how I could do this with SDL_gpu?

[ephemer]

Ok, so I figured out a way of doing what I need as follows:

in vec4 originalColour;
in vec2 absolutePixelPos;

out vec4 fragColor;

uniform float maskMinX;
uniform float maskMinY;
uniform float maskWidth;
uniform float maskHeight;
uniform sampler2D maskTexture;

void main(void)
{
    vec2 maskCoordinate = vec2(
        ((absolutePixelPos.x - maskMinX) / maskWidth),
        ((absolutePixelPos.y - maskMinY) / maskHeight)
    );

    vec4 maskColour = texture(maskTexture, maskCoordinate);
    fragColor = vec4(originalColour.rgb, originalColour.a * maskColour.a);
}

... with absolutePixelPos just being the gpu_Vertex passed into the default vertex shader and e.g. maskMinY being the absolute Y position of the masked shape. This works, but it feels weird to be working with in absolute texel positions (including maskMinY) etc.

I noticed that DoUntexturedFlush doesn't set the texcoord AttribArray like DoPartialFlush does. If it did (and it's quite possible I'm misunderstanding this), I get the impression these absolute pixel positions could go away (including the maskWidth/Height) and I'd be left with the very simple code snippet I posted a few days ago. It seems valuable to allow shaders to work on shapes generally, so would you accept a change along those lines in a pull request?

Or do you have a better idea? This is the first shader I've ever written, so I'm sure there are much cleverer ways to go about all this.

[ephemer]

I am going to try to change the render code to use the matrix stack instead of blitting absolute positions, I think this may fix the issue I'm having. Sorry for spamming, but this may be useful for someone else having the same issues.

[RandomErrorMessage]

On the topic of stencil buffers I've been trying to track down some issues relating to NanoVG and I suspect SDL_gpu lacking a stencil buffer might be the cause. Am I wrong for thinking there is no stencil buffer initialized? Would it be possible to do this without major code changes, unrelated to the GPU_Image struct itself.

EDIT: nevermind, you can set SDL_GL_SetAttribute(SDL_GL_STENCIL_SIZE, 1); before creating the context