Implement per-object shadows to improve the quality of received shadows on specific objects

[Lielay9]

Describe the project you are working on

3rd-person (very) small open world -action game in NPR-style.

Describe the problem or limitation you are having in your project

The quality of close-up dynamic shadows isn't good without increasing shadow resolution. However, the extra resolution is spent wastefully decreasing the performance unnecessarily. Often the shadows of the environment could be rendered at a lower resolution but the resolution has to be set high to get passable shadows for characters. This is especially true for the self-shadows on the characters and smaller detail.

Most notably, a global shadow map is inefficient when used with baked lighting since only a few dynamic objects are left to light/shadow. This greatly diminishes the effectiveness of baking shadows in the first place.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

By rendering dynamic meshes to their own shadow map-atlas, they could benefit from better-quality shadows. This gives agency to the developer to up the effective shadow resolution where it's most needed (e.g. player-character, certain enemies). This also considerably increases the performance when used alongside baked lighting/shadows since the resolution of the shadow map is used more efficiently for the remaining dynamic objects.

When used alongside a traditional global shadow map (i.e. no baked static shadows), per-object shadows do have a performance hit. That said even with mainly dynamic lighting, the overall cost might be lower if it avoids the need to use a higher-resolution shadow map. For reference here's an extract from the CryEngine documentation about the cost (link):

... The drawbacks are increased memory consumption of the additional shadow maps and increased shadow filtering cost. As an example, the additional cost amounted to approximately 0.25ms for the main protagonist in Ryse on XboxOne.

Here are some examples from Ryse to better quantify the quality the 0.25ms brought:

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

At least CryEngine and Unreal Engine support per-object shadows (In Unreal land, per-object shadows are called inset shadows). In Unreals case they're enabled by default for dynamic objects. Couldn't find that good details about their implementations but here are couple of links that might be useful:

• Unreal 3? not very specific but outlines the technique (also some nice pictures): link

• Third-party explanation of Unreals implementation in Chinese (Probably most useful): link

• The CryEngine doc (again, and pretty useless): link

• Siggraph 2013, CryEngine (mainly pretty pictures): link

• Someone's implementation in Unity (More Chinese): link

Some snippets from above:

• Objects marked to render per-object shadows work by rendering a depth buffer for the object from the perspective of the light with the help of their bounding box and then projecting it onto the world.

• For animated skeletal meshes it may save time computing a new tighter bounding box every frame.

• The per-object shadows could/should be only used for directional lights since they're the main culprit for low-resolution shadows.

In addition to relevant project settings etc. it would be useful if shaders could customize what layers they sample and probably also how much; only cast or only receive per-object shadows or blend with the global ones etc.

If this enhancement will not be used often, can it be worked around with a few lines of script?

Not without modifying the rendering pipeline.

Is there a reason why this should be core and not an add-on in the asset library?

Cannot be done without modifying the rendering pipeline. Guess you could write your own renderer :^)

[Lielay9]

@clayjohn I've changed the proposal to better outline how per-object shadows can be used to increase performance when used with baked shadows. Thus, should this also warrant a topic:performance? I just like the color of the label.

[Calinou]

If you want detailed shadows for dynamic objects but still want far away static objects to cast shadows, shadowmasking is probably a better solution. It allows you to greatly reduce your DirectionalLight3D's Shadow Max Distance property without affecting how static shadows look in the distance. This can actually improve performance, as fewer objects need to be included in the shadow mapping draw calls.

Per-object shadows seem like a highly AAA-oriented solution to me. I don't think this kind of solution would see a lot of usage in Godot, considering how demanding it is (and how it needs highly detailed models to really be beneficial, unlike shadowmasking which benefits almost any scene out there). We should probably be looking into ways to make shadow rendering cheaper in the first place :slightly_smiling_face:

Godot 3.x used to support contact shadows, which trace pixel-perfect shadows in screen-space (this is intended for contact hardening). These were very expensive though, and regularly caused GPU lockups to occur if the camera got too close to a mesh. Contact shadows were removed in 4.0 in favor of shadow normal offset bias/shadow pancaking, which provide similar contact hardening capabilities but with a much lower performance cost.

[Lielay9]

If you want detailed shadows for dynamic objects but still want far away static objects to cast shadows, shadowmasking is probably a better solution. It allows you to greatly reduce your DirectionalLight3D's Shadow Max Distance property without affecting how static shadows look in the distance. This can actually improve performance, as fewer objects need to be included in the shadow mapping draw calls.

This doesn't really seem to address the same issues though. Shadow masking isn't from the looks of it able to do much unless the dynamic objects are neatly next to each other. That's not the case for mainly stationary objects that still need to be animated, such as windmills, some trees, street lamps, destructible objects and openable doors etc. Frankly, for most of the environments I've worked with, I don't think shadow masking would be that helpful.

Per-object shadows seem like a highly AAA-oriented solution to me. I don't think this kind of solution would see a lot of usage in Godot, considering how demanding it is (and how it needs highly detailed models to really be beneficial, unlike shadowmasking which benefits almost any scene out there). We should probably be looking into ways to make shadow rendering cheaper in the first place 🙂

Individual shadows can be more expensive if used with dynamic lighting. But with mostly static lighting they give higher fidelity shadows for a much better performance than traditional one-size shadow map. Not only that but they are not affected by the caster's distance resulting in better shadows far away, and sometimes constructing CSM can be omitted altogether. I would make the bold claim that all games in Godot using baked lighting would benefit from this feature with both increased performance and shadow quality.

[clayjohn]

A character/per-object shadow map system is something I would like to eventually see in the engine. For many games, you can get away with a much lower shadow resolution if your character uses its own shadow map which can result in big wins for performance and quality.

[yosoyfreeman]

I think that seeing per object shadows are really helpful not only for detailed objects but for smaller ones, or things that have to be really close to the camera.

[ODtian]

surely should be implemented in godot, at current stage pssm give good shadow in large scene but not good at character/object self-shadow. This is fatal in non-physical rendering because it often uses smooth hard shadows to create a sense of volume.

[Calinou]

A character/per-object shadow map system is something I would like to eventually see in the engine. For many games, you can get away with a much lower shadow resolution if your character uses its own shadow map which can result in big wins for performance and quality.

When you're referring to per-object shadows, are you only referring to self-shadowing or also casting a shadow on the floor below the character? The implementation likely differs a lot between those two.

[Lielay9]

When you're referring to per-object shadows, are you only referring to self-shadowing or also casting a shadow on the floor below the character? The implementation likely differs a lot between those two.

@clayjohn can correct me if they had something else in mind, but the proposed per-object shadowing system (as seen in cry engine and unreal) in its simplest form works by creating a shadow map per object (atlas if there are many) instead of a one big one. It's practically the same as how point- and spotlights work (except only shadows are local, light is still global). So ground shadows are indeed included.

I actually created an unpolished implementation of this ~month ago which yielded promising results (i.e. it didn't crash immediately) and thought of many ways it could be improved. I will finish it at later date and use it in my own project since I need to know which object cast the shadow --- unless a better solution emerges before then.

A simple demonstration (no shadow-bias).

Edit: you can also reference the ps3 title Final Fantasy XIII (2009), that used the technique (characters have separate shadows) https://twitter.com/AmazingThew/status/1579256188537278465. Image from link (left=character shadows, top-right=environment shadows, bot-right=depth):