Restore a workflow for light masks

[markdibarry]

Describe the project you are working on

A 2D RPG with faux light sources and cutouts.

Describe the problem or limitation you are having in your project

In traditional pixel art games, for spotlights and cutout-follows you'd use a mask and an overlay. This was easy to do in 3.x by adding a Light2D node as a child of your character and setting it to mask mode. This would allow the user to have a cutout-follow that looked like your character is giving off light.

In 4.x Light2D was replaced by PointLight2D, and the mask mode was removed. To achieve the same result as above, it now requires separation of the masks and their sources, spread across the scene tree, needing the masks to be direct children of the overlay, and either a RemoteTransform2D assigned to each mask or managed in scripts, shown below.

https://github.com/godotengine/godot-proposals/assets/21325943/bd0d38dc-35f8-4e7c-894d-83c4c61acf2e

Describe the feature / enhancement and how it helps to overcome the problem or limitation

There are multiple benefits to restoring a workflow where the masks and the targets aren't separated.

• It removes the need to design and write a system for each project to accommodate adding targets to a scene, and managing the separate masks, cleaning them up when removed, etc.
• Faster, more user friendly UX
• Visually it better displays the relationship between a "light" and light source/target.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

Ideally it'd be a similar workflow to 3.x where you can just designate a layer that would be effected by the mask, and set the overlay to that layer.

If this enhancement will not be used often, can it be worked around with a few lines of script?

It can be worked around, but not with a few lines. It currently requires writing an entire management system for any scene that needs it.

Is there a reason why this should be core and not an add-on in the asset library?

This existed prior to 4.x, and some may consider it fixing a regression rather than feature proposal.

[markdibarry]

I was able to create a somewhat scalable workflow for a light-mask/cutout shader, but I feel like there should be a better way. It requires a build off of this PR with the new clip_children subtract mode.

1. Adding a CanvasGroup as a child of the CanvasItem you want to mask.
2. Adding an all-white ColorRect as a child of the CanvasGroup
3. Enabling the Light Mask layer of the light on the ColorRect
4. Keep the ColorRect the same size and position as the target. note: you should now see the light as black on the all white ColorRect
5. Overwrite the CanvasGroup built-in shader with this:

   
   uniform sampler2D screen_texture : hint_screen_texture, repeat_disable, filter_nearest;

void fragment() { COLOR = texture(screen_texture, SCREEN_UV, 0.0); if (COLOR.a > 0.0) { COLOR = vec4(COLOR.rgb, 1.0 - COLOR.r); } }

6. This will render the lights only as black, with no background and preserve the alpha.
7. Set the parent `CanvasItem` (`Sprite2D`, `ColorRect`, etc), to `clip_children` subtract.

https://github.com/godotengine/godot-proposals/assets/21325943/6b73a0f4-aabf-4b02-a6ee-7f81258b9162

[markdibarry]

Hm. After some more testing, my last comment seems like not a great solution after all, since you can have only 16 lights total which you'd realistically bump up against pretty easily (imagine only being allowed 16 torches/actors/etc in a level total).

So far, all other non-light workflows requires scripting a tightly coupled system where the masks need to be separate from the mask owner and everything needs to be aware of one another. I'm just looking for a workflow that is dead simple like light's one-to-many "I'm on Layer 2, so I get sent to everything on Layer 2", but doesn't have the 16 mask limitation. Or even being able to send a bunch of textures to a viewport or layer without having to be direct children of that viewport/layer just like Lights allow. I guess lights are also supposedly a huge performance tank, but I've yet to notice.

[SlugFiller]

If you want to draw into a different buffer without adjusting the scene tree, this proposal is an option: https://github.com/godotengine/godot-proposals/issues/6506

However, if you want >16 lights/sprites, the repeated context switching would be painful performance-wise.

Use of visibility layers can be combined with a world-inheriting viewport to get a similar effect without the cost.

Of course, the best solution might actually be to handle it in script: Put all the "to be drawn elsewhere" nodes inside an invisible node with a certain group. Then, use get_tree().get_nodes_in_group to get all the nodes, and clone them to a certain parent (e.g. a white rect with subtract-children). Do this on _process, which is how RemoteTransform2D does it anyway. You can recreate all the cloned nodes in the target parent each frame, or you can add some map cache so that nodes don't need to be re-created (might improve performance).

[markdibarry]

@SlugFiller Proposal: With that proposal, would it still be painful if there were 16+ sprites writing to the same buffer? I basically just want to build up a single mask texture from all the sources to pass around to things like overlays and assets that should have cutouts.

16+ lights/mask limit: honestly don't prefer lights at all, it's just a vastly superior UX workflow and seems to be the only way I can make a faux light mask or similar effects without having to script a system any time I need them. Handling it in script is always an option, but it winds up very tightly-coupled, complex, brittle, and error prone. Plus cloning a bunch of objects and managing them separately sounds like a nightmare to manage gracefully and, as you mentioned, probably a performance ditch. When you offload it to scripting a tool/plugin/system it also begs a lot of questions:

• "How do you know which object should have a mask and how big/how should it behave?"
• "Should this system be included for every scene?"
• "If so, which objects should the system check? Who should do the checking to prevent exceptions?"
• "How should it be checked and how often?"
• "How can you prevent this from being a custom-system per game?"
• "Does it need to maintain a list at all times of these as they're added to the scene?"
• "How reasonable is it to duplicate the mask on every overlay/object that needs masked?"
• "How should cleanup be handled to prevent memory leaks?"
• "Should the masks be pooled? Can they pooled if they'll need to be duplicated?"

Visibility layers: That sounds pretty interesting. When I first saw visibility layers, I thought that may be an alley to explore, but I couldn't find any documentation on it and in quick tests it didn't seem to work as an actual layer, but more a visibility group as a shortcut for the visibility property. Anything more you could share about that idea?

[SlugFiller]

With that proposal, would it still be painful if there were 16+ sprites writing to the same buffer?

The issue, moreso on mobile than on desktop, is not how many buffers you write to (Canvas groups use the same buffer if on the same level, even with my PR), but rather, how many times you switch between writing to one buffer, and writing to a different one. If you basically have an entity, that's made from one sprite and one light, each writing to a different buffer, and then you have 20 of those entities, then what happens is that you draw sprite buffer->light buffer->sprite buffer->light buffer->sprite buffer->light buffer. Each such switch between target buffers is extremely slow on mobile GPUs. So much that you don't want to do more than 5 or 6 of them, let alone 20. On Desktop, it's faster. But it's still way faster if you do all the draws to a given buffer in a row.

it also begs a lot of questions:

I think almost all of those questions can be answered with: A script can export properties. Have one property be the group name of "mask nodes", and a second property be the parent node into which they are cloned (You can export node properties now, so it doesn't even need to be a NodePath), and bam. You have a script that you can use in any situation, in any game.

For extra credits, instead of having it duplicate actual nodes, have a Node2D with metadata specifying a scene name, and have it instantiate that scene instead of duplicating the node. Additional metadata could be used to send properties to the scene, so you can have some extra customization.

Visibility layers

Hmm, in 3D, a viewport inside a scene, unless set to Own World 3D, will render the same scene that it is in. This makes it easy to draw the same scene from different camera angles, or with different cull masks (to hide/show certain objects in certain viewports).

However, I looked again, and couldn't find a clean way to draw the same 2D scene in two different viewports. Maybe it deserves its own proposals. Although it might not require a viewport. Since this can be handled at the canvas item sorting level, there's no reason not to outright copy a subtree filtered by a certain cull mask. Maybe have a special node that does that specifically. It would need to have some limiations applied to it (like only copying a node from earlier in the tree, must copy the same clip size and global transform), but is otherwise doable.

[markdibarry]

@SlugFiller Okay that makes sense. I had already heard it's generally advised to avoid using light nodes or CanvasGroup/clip_children if you can help it, since it tanks performance, which is a shame since I think CanvasGroup is probably most used for composite sprites, which would require one for every sprite. I totally understand the limitation though. Sounds like the whole system would have to be rewritten to get them usable in a 2D project.

I guess in the meantime, if there's no way around having to make a separate system just for this effect, I imagine the most performant solution solution is to try to mirror the SNES layer setup: Offload the work to the CPU by making a SubViewport just for the sprite masks, and manage their positions and zoom based on the camera position. Then pass the viewport as a TextureRect to things needing to be masked.

I'm not sure of the performance of a separate SubViewport, but I can't imagine it'd be worse on mobile than the situation you described.

[markdibarry]

@SlugFiller Just an update. After you commented about the limitations and why what I wanted to do would switch contexts too much, I spent some time working up a few solutions, and the only viable one was a MultiMeshInstance2D solution. It works by applying the gradient as a mask, dithering it and using a backbuffer copy before the CanvasItem(s) you want to mask. Then you have a node that just has basic data like the position, light size, and speed which represents the instance and send it to a "Light hub" that updates the multimesh. Of course it requires a second/different hub with an offset if used with a CanvasLayer overlay or something, but I was surprised at the results. Despite it being the only viable solution, it does a great job! I tried with Forward, Mobile, and Compatibility and got almost no fps drop between 8 and 8000 lights, still allowing different bayer patterns, sizes and flicker speeds for each light. Here it is with 8:

https://github.com/godotengine/godot-proposals/assets/21325943/c6ce3f65-c560-492a-9f83-db4c8b8afd33

Here it is with 8000:

https://github.com/godotengine/godot-proposals/assets/21325943/cd4f71df-8895-4cb9-8951-d82ae4cb5d7d

I saw that it said there could be a performance drop if the mesh instances were far apart, but I tested by spreading them out randomly on a canvas of 500,000 x 500,000, and didn't see any difference in performance. Not sure if you can speak to that? If there's no issues, I'll close this ticket, since there's a workaround and I understand it's just a weak point of the engine.

[markdibarry]

Since there is no comment, I'm going to assume the documentation is outdated and incorrect regarding the performance warning about mesh instances being far apart. I'll make a separate PR to remove the warning from the documentation.