[SOLVED] Mask Regions display as Black when using Transparent Window (Clip Children, and hint_screen_texture w/ BackBufferCopy + Shader)

[ImamuraCross]

Tested versions

Reproducible in:

• v4.2.2.stable.mono.official [15073afe3]
• v4.3.stable.mono.official [77dcf97d8]
• v4.3.stable.official [77dcf97d8]
• v4.4.dev3.official [f4af8201b]

The issue occurs on Forward+, Mobile, and Compatibility Rendering.

System information

Godot v4.3.stable.mono - Windows 10.0.19045 - Vulkan (Forward+) - dedicated NVIDIA GeForce RTX 2060 (NVIDIA; 32.0.15.6603) - AMD Ryzen 7 5800X 8-Core Processor (16 Threads)

Issue description

Issue only observable when Running Project; not in Viewport.

When masking with Clip Children (Clip Only) or via hint_screen_texture w/ BackBufferCopy, areas of the mask that do not overlap with anything appear as black, instead of simply not showing up, which is expected and preferable. Otherwise, the mask works perfectly.

I assume some kind of premultiplied alpha error, though I'm no expert.

Issue confirmed to occur when Polygon2D, Sprite2D, and Label Nodes are used for masking; I've not tested anything further.

Issue not observable in Viewport

Running Scene: Window is transparent, as expected, but Masking Regions are now visible; unwanted

With ColorRect background, to show that black regions are only visible when overlapping nothing

Steps to reproduce

The following settings were used: Project Settings -> General -> -> Display -> Window -> Size -> Transparent: On -> Display -> Window -> Per Pixel Transparency: On -> Rendering -> Viewport -> Transparent Background: On

• Clip Children Nodes are set to (Clip Only).
• The BackBufferCopy's Copy Mode is set to Viewport; the Shader Code I used for the mask is as follows:

  
  shader_type canvas_item;

uniform sampler2D screen_texture: hint_screen_texture;

void fragment() { COLOR = texture(screen_texture, SCREEN_UV); COLOR.a = texture(TEXTURE, UV).a; }

----

Simply open and run the MRP; the issue should be immediately observable.

MRP only contains an example of the **hint_screen_texture w/ BackBufferCopy** approach with Polygon2D, but I've confirmed the issue also occurs with the aforementioned Node types (**Sprite2D**, **Label**).

Enabling visibility on the **"greenbg" ColorRect** and moving it will demonstrate that the issue only occurs if there is nothing below the masking elements.

### Minimal reproduction project (MRP)

[masking_issue_test.zip](https://github.com/user-attachments/files/17642019/masking_issue_test.zip)

[clayjohn]

I think the clip_children mode (and your own shader) needs a different version for handling transparent backgrounds. The current clip_children shader is:

shader_type canvas_item;
render_mode unshaded;

uniform sampler2D screen_texture : hint_screen_texture, repeat_disable, filter_nearest;

void fragment() {
    vec4 c = textureLod(screen_texture, SCREEN_UV, 0.0);
    COLOR.rgb = c.rgb;
}

As you can see, it takes the alpha of the parent, but the color of the child. Which means that when the background is transparent black you end up reading plain black.

For the case of a transparent background, simply doing a pre-multiplied alpha might be enough (e.g. COLOR.rgb = c.rgb * c.a;). I would need to investigate more deeply to know if that would cause any issues for the non-transparent background case

[ImamuraCross]

Thank you for such a swift reply! I wasn't aware that the clip_children toggle is enabling a shader; that's very interesting.

I see what you mean, though, regarding the black; I was afraid of that. I suppose it's less of a bug, and more so me misunderstanding how the transparent window actually works.

I had assumed this kind of premultiplied alpha operation (as well as similar shaders like Add, Multiply), would take into account whatever's beneath the game window, but I appear to have been mistaken there.

---

I should note that I only have a couple months' experience in Godot; while I'm comfortable with GDScript, the shader language is beyond me for now. I should have mentioned that the code I used above was taken from online, and that I didn't fully grasp its workings. My apologies :')

To make sure I understand what you're saying...

void fragment() {
    vec4 c = textureLod(screen_texture, SCREEN_UV, 0.0);
    COLOR.rgb = c.rgb;
}

... I'm assuming that vec4 c is the equivalent of var c : Vector4, and that textureLod(screen_texture, SCREEN_UV, 0.0); is an RGBA value, of which only the RGB is being set to 'COLOR'.

I think the value is being taken from the child...? Or perhaps from what's already been rendered on-screen; I've just skimmed the Screen-reading shaders page; I'll be giving it a deep read soon.

I tried using the code you posted in Godot, but I'm having difficulty reproducing the clip_children effect. I'll have to learn more about the shader language and test out your suggestion of doing a pre-multiplied alpha; I think I understand what you're suggesting, conceptually, at least.

---

In the meantime, my workaround for masking Polygon2D objects specifically has been using Geometry2D boolean operations to update the object's polygon array every physics_frame (when necessary); that seems to work well, and I recommend that to anyone reading the thread!

Thank you again! It's my first time posting on GitHub, so I hope I'm following proper posting/discussion etiquette as well as possible. I appreciate your patience and time, and hope you're having a fantastic day :>

[ImamuraCross]

Okay, I spent the day learning shader language basics, and have found a solution!

To anyone who needs it...

Masking objects with a transparent window

What's working for me is using something called a SubViewport node; in particular, its Viewport Texture property.

The example below uses mask_1 as the mask data to mask the 4 Icon nodes below it. I'll run through my process and explain things best I can! IT WORKS WHILE RUNNING YEAHHH

SubViewport Setup

• The SubViewport is a object container of sorts; the cool thing is that its contents only render internally; they do not show in the final render of your game. When creating it, make sure to set the size to your game's resolution, as we'll be using the screen's UV coords!
• SubViewports' contents do not show up in the Viewport, either, meaning the objects inside cannot be dragged, rotated, scaled, etc. They can be viewed and edited through the Inspector, however.
• Another option is to (temporarily) place the SubViewport node inside a SubViewportContainer; this will cause it to show up in the Viewport (but will also have it show in the final render!)

Mask Setup

• As you can see above, I've only placed black and white objects in the SubViewport. I'm using white to indicate where masked objects show, and black for where they are hidden. You do also need a backing color, either black or white; do not leave any empty areas!
• For the Sprite2D, which had an existing texture, I wrote a very simple shader to convert it to flat white (or black, if needed):

  
  shader_type canvas_item;

uniform float value: hint_range(0.0, 1.0, 1.0);

void fragment() { COLOR.rgb = vec3(value, value, value); }

## Masking Shader
- I wrote this shader for the objects to be masked; it takes the `mask_texture` (which I'll cover in the next section), extracts the black/white values, and converts them into alpha values!

shader_type canvas_item;

uniform sampler2D screen_texture: hint_screen_texture; uniform sampler2D mask_texture; uniform bool white_reveals = true; // Black hides. When false, black reveals and white hides. uniform bool clip_blending = false;

// 0: MIX, 1: ADD, 2: SUB, 3: MULTIPLY, 4: SCREEN uniform int blend_mode : hint_range(0, 4);

void fragment() { // COLORING switch (blend_mode) { case 1: COLOR.rgb += vec3(texture(screen_texture, SCREEN_UV).rgb); break; case 2: COLOR.rgb -= vec3(texture(screen_texture, SCREEN_UV).rgb); break; case 3: vec3 screen_tex = vec3(texture(screen_texture, SCREEN_UV).rgb); if (screen_tex.rgb != vec3(0.0, 0.0, 0.0)) { COLOR.rgb = screen_tex; } else if (clip_blending) { COLOR.a = 0.0; } break; case 4: vec3 screen_tex = vec3(texture(screen_texture, SCREEN_UV).rgb); if (screen_tex.rgb != vec3(0.0, 0.0, 0.0)) { COLOR.rgb = 1.0 - (vec3(1.0, 1.0, 1.0) - vec3(texture(screen_texture, SCREEN_UV).rgb)) (vec3(1.0, 1.0, 1.0) - COLOR.rgb); } else if (clip_blending) { COLOR.a = 0.0; } break; }

// ALPHA MASKING
if (white_reveals) {
    COLOR.a *= (
    texture(mask_texture, SCREEN_UV).r
    + texture(mask_texture, SCREEN_UV).g
    + texture(mask_texture, SCREEN_UV).b
    ) /3.0;
} else {
    COLOR.a *= 1.0 - (
    texture(mask_texture, SCREEN_UV).r
    + texture(mask_texture, SCREEN_UV).g
    + texture(mask_texture, SCREEN_UV).b
    ) /3.0;
}

}

- The `white_reveals` toggle is for if you want to use black to reveal, and white to hide. This is in case you want two objects to use the same masking data, but to be masked in opposite ways, for instance.
- The `blend_mode` is there as a little bonus :> I couldn't find a way to set a shader parameter as a drop-down, so please refer to the comment below for what setting does what; it blends with everything below it, like the default `CanvasItemMaterial` options do. HOWEVER, my shader prevents the floating black shapes issue by only applying the blending to overlapping areas; any part of the object that does not overlap anything simply has its base color retained.
- The `clip_blending` ONLY want overlapping parts to be shown! Comparison below:
![image](https://github.com/user-attachments/assets/306fe2ef-10bf-4328-9b99-c40d35b9dc82)

## Script for Masked Objects
As noted above, the `mask_texture` has to be passed through to the shader for all of this to work; we'll be grabbing the SubViewport's Viewport Texture.
This only has to be done once; you can even **animate** mask_1's contents to have moving masks and the like; super fun!!
```gdscript
func _ready() -> void:
    await RenderingServer.frame_post_draw # Ensures that the Viewport Texture is ready to go before the next line runs
    material.set_shader_parameter("mask_texture", $'..'/mask_1.get_texture())

Final Notes

This is my first time using shaders, so please feel free to inform me on anything that could be optimised/better structured! I'll leave the thread open for any feedback or questions.

In particular, I'm not sure how to deal with the aliasing when using blend modes; I'm sure there's something that can be done, but I lack the knowledge at present...

Take care, and good luck on your projects!