Vulkan: Strange colors appear on start due to ReflectionProbes not being initialized correctly

[davidak]

Godot version

4.0.dev.20211004.official.2e8cba0bd

System information

NixOS 21.11.git.37cc619178fM, AMD RX 480, amdgpu, Vulkan

Issue description

https://youtu.be/EcDbaj71MuE?t=7

Downstream issue: https://git.gieszer.link/unfa/liblast/issues/92

Steps to reproduce

1. Download project
2. run ./Liblast\ Linux

Minimal reproduction project

https://github.com/davidak/test/releases/download/0.1.0/Liblast.0.1.0.1-pre-alpha.zip

[unfa]

This issue is caused by Reflection Probes. I think that they initialize with random data used as their cubemap textures or as incorrect data (NaNs etc.), before th ecubemaps are rendered. One can see that the Reflection Probles are updated one by one and after the last one is done, the image looks as expected.

I think there's a need to be able to manualy trigger a Reflection Probe to render it's cubemap and to be able to store/restore/change this data via scripting. In another project based on Godot 3 I've implemented level streaming and the momet the next chunk was spawned, the whole game froze to wait fro the cubemaps fro Replection Probes to be rendered. I had to remove them all together, as there was no way to cache and supply that data beforehand.

Godot 4 seems to be removing the wait time, so that new frames can be rendered, but this creates a different problem of messed up visuals while the Reflection Probes are not initialized yet.

[Calinou]

Godot 4 seems to be removing the wait time, so that new frames can be rendered, but this creates a different problem of messed up visuals while the Reflection Probes are not initialized yet.

Initializing the probe data to black (or perhaps the sky color) would likely resolve this issue in a "good enough" manner.

The issue with implementing offline storage for ReflectionProbe is that it potentially requires a lot of storage space, and is incompatible with probes that need to be moved around or updated in any way. The data also needs to be regenerated and saved every time you move the ReflectionProbe in the editor. (We can't do this before exporting the project, as the scene may not be open in the editor while exporting the project.)

While AAA games generally use offline rendering for static reflection probes, I don't think it's viable for most indie games due to the additional bake times, manual work and storage space needed.

As a workaround, use a ColorRect to initially hide the 3D viewport for 0.5-1 second (and progressively fade it using an AnimationPlayer). This will also hide SDFGI and volumetric fog reprojection at the same time.

[unfa]

Maybe there could be a way to automate the process with some metadata and for example have the editor open scenes containing ReflectionProbes with "Cache Data" enabled, render those to textures and package with the game export?

If storage is a problem, then the games could fro example bundle low resoluton versions to initialize the probes with then after the high-resolution is done rendering it could be swapped or better - blended over some time to avoid a visual jolt.

[unfa]

But surely having it just pure black would be good enough for a quick fix.

[atirut-w]

In Unity, IIRC, there is an option to either have the probe update in real time or baked. Not sure what would be a good way to implement it without breaking compat, though.

[unfa]

I've asked for an option to bake the probes beforehand, but that was deemed too problematic.

I would love to have low-level access to it's cubemap texture so I can store and restore it.

Right now in Liblast a level loads in a fraction of a second to 2 seconds, but then the user has to wait like 10-15 seconds to get all the reflection probes baked. Which I do by waiting some frames (so far 6 * 10 per probe seems to work, but that's 1 second of waiting per probe.

And sometimes the probes still don't manage to bake out during this time and then the game starts, and one of them is borked, because a character spawning illuminated the map while the reflection probes was baking.

If there was a signal sent that'd let me know when a reflection probes has finished baking, I could at least make this work 100% of the time.

If additionally I had an option to save and load the cubemap, I could design a process to pre-bake the probes in my game and avoid this long wait time. Maybe not even ship the prove textures, but bake them once client-side and not have to re-do that every time a level is loaded.

Having reflection probes have to be baked every time from scratch becomes quite problematic with even slightly more complex game levels, and I don't know how that'd go for an actually large one.

We can pre-bake light maps and Voxel GI, why not Reflection probes too?

[clayjohn]

Has anyone been able to create an MRP for this issue? It would be helpful in identifying where the bug is coming from.

@unfa I think pre-baking reflection probes is a good idea. Users will just have to be made aware of the memory trade off made when doing so (probes take a lot of storage space: size 1024 would be about 66mb while a size of 256 would be about 4 mb). There is already a proposal about improving the probe baking/update process so I don't think we should derail this bug report with a discuss about pre-baking probes, that discussion is better had in a proposal

[ArtyIF]

I noticed that on Intel's iGPUs (in my case Godot v4.2.beta2.mono - Fedora Linux 39 (Workstation Edition) - Wayland - Vulkan (Forward+) - integrated Intel(R) Graphics (ADL GT2) () - 12th Gen Intel(R) Core(TM) i3-1215U (8 Threads)) the probes are initialized all-black, while on AMD's GPUs (in my case Godot v4.2.beta2.mono - Fedora Linux 39 (KDE Plasma) - Wayland - Vulkan (Forward+) - dedicated AMD Radeon RX 570 Series (RADV POLARIS10) () - AMD Ryzen 5 2600 Six-Core Processor (12 Threads)) it's the colorful mess that's shown on the screenshot. Not sure about NVIDIA and non-Linux platforms

[ArtyIF]

Has anyone been able to create an MRP for this issue? It would be helpful in identifying where the bug is coming from.

Try this one: VkBugRepr.zip I didn't get a chance to test it on AMD yet. I also set run/max_fps to 5 so that the probe took longer to update

[ArtyIF]

Yeah, it should work on AMD, at least it did on my setup.