Improve appearance of screen-space reflections close to screen edges by bending reflection rays

[Calinou]

• Related to https://github.com/godotengine/godot-proposals/issues/3854 and https://github.com/godotengine/godot-proposals/issues/3919.

Describe the project you are working on

The Godot editor :slightly_smiling_face:

Describe the problem or limitation you are having in your project

Screen-space reflections currently fade towards the sky at the edges of the screen to make artifacts less noticeable. This works fine in outdoor scenes with little geometry, but shows its limits in indoor environments such as caves, houses with reflective floors, and so on.

In the video below, focus on the left and right edges of the screen:

https://github.com/godotengine/godot-proposals/assets/180032/5b5bbb44-60b1-4f3e-ad00-ed6aab4485a1

Testing project: test_ssr_fade_edges.zip

Describe the feature / enhancement and how it helps to overcome the problem or limitation

By bending reflection rays from screen-space reflections when they are near the edges of the screen, we can ensure the edges of the screen have some reflection visible instead of fading towards the sky (which can look obvious with large flat planes of reflective materials).

There's still some visible distortion occurring with this approach compared to "ground truth" reflections (such as planar reflections), but it's generally far less noticeable than fading towards the sky.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

Quoting from the Brickadia article linked above:

Essentially, what we want to accomplish is to make sure the reflected rays stay within the view frustum. To do this, we have to switch to a different coordinate space.

Imagine a cube. It has an extent of one unit on each axis, and is centered at the origin. If you draw a top view of this cube, with the coordinate system overlaid, it looks somewhat like the below image. You could consider this something like cube-space, and a set of coordinates between -1 and 1 on each axis would let you position something anywhere relative to the cube.

We can do the same thing for the view frustum of our camera above, except we stretch it, like the below image. You can now use coordinates between -1 and 1 to position something anywhere within the view frustum.

In graphics programming, this is normally called "clip space", because objects outside of the unit cube will be clipped by the edges of the screen. The part we care about though, is that parallel lines starting inside the view and going "forward" in this coordinate space will remain within the view. We can visualize our reflection ray going off screen (red) in this coordinate space.

All we have to do to ensure we get a valid reflection point, is align it to face straight forward in clip space. This is as easy as ensuring the x component of its direction is equal to 0.

[...]

One last try before we give up, what if we kept this effect but only applied it towards the edges, fading it out towards the center where most of the attention goes?

The final implementation would look something like this (focus on the left and right edges):

https://github.com/godotengine/godot-proposals/assets/180032/3dd1b2af-d5a3-4d60-830d-df0fdae04e81

Video credit: Brickadia

If this enhancement will not be used often, can it be worked around with a few lines of script?

No.

Is there a reason why this should be core and not an add-on in the asset library?

The screen-space reflection implementation is part of core and can't be modified by an add-on.

[Usnul]

Looks really trippy. I don't think this is a good idea, the effect is incredibly noticeable.

SSR is just like that, you're working with an incredibly limited data set (what's on screen).

The best solutions for now are:

1. fade out SSR towards the edges of the screen (most common)
2. fill samples with environment map samples (pretty common, Unreal does this out of more prominent examples)
3. retrace samples that go off-screen using another technique, typically this would be some kind of full scene ray trace. (rare-ish, but Unreal does have this again).

[Calinou]

1. fade out SSR towards the edges of the screen (most common)

This is what Godot 3.x and 4.x do currently. It really shows its limitations in scenes where geometry is present on the sides of the screen and the sky color is highly different from the solid geometry (e.g. an indoor cave with a lake).

While not perfect, I consider these stretched reflections to be a better alternative in the vast majority of situations.

2. fill samples with environment map samples (pretty common, Unreal does this out of more prominent examples)

Cubemap fallbacks are already feasible in Godot, but they can't be used in all scenarios depending on the scene topology.

3. retrace samples that go off-screen using another technique, typically this would be some kind of full scene ray trace. (rare-ish, but Unreal does have this again).

Godot doesn't have raytracing support yet, but when it's implemented, it won't run with usable performance on all GPUs (or not with the desired FPS target).