reflection probe resoloution never changes despite changing the reflection atlas if reflection probe is in UPDATE_ALWAYS mode

[IPlayKindred]

Tested versions

reproducible in godot 4.0, godot 4.1, godot 4.2, godot 4.3 dev 1 to dev 6.

System information

windows 10 and linux mint version 21.3

Issue description

changing the reflection atlas size does not change the reflection probes resoloution. changing the reflection atlas size to anything above 512 (1024 for example) just breaks them all, makes them black and ruins the games performance.

Steps to reproduce

csg cube, reflection probe, put reflection probe inside cube, go to settings, go to reflections, go to reflection atlas, change the size to anything.

Minimal reproduction project (MRP)

https://drive.google.com/file/d/1ixRT9pDb_5yXnJ0zw3WRlbD1_Cf_mEJX/view?usp=sharing

[AThousandShips]

What resolution are you talking about for the probes? There's no such property or method

[IPlayKindred]

the one in the project settings, under reflections, it says changing it changes the cubemap face size, ie resoloution of the cubemaps.

however doing so either breaks them (anything above 512) or does nothing, they stay on 256 resoloution no matter how lower or higher you go.

there's a way to change individual cubemap resoloutions, however it does not work because of the before mentioned issue.

p.s : I use the terms "cubemaps" and "reflection probes" interchangeably since they both mean the same thing to the players anyways.

[AThousandShips]

I use the terms "cubemaps" and "reflection probes" interchangeably since they both mean the same thing to the players anyways.

Good clarfification, but since there are reflection probe nodes I think being clear about what you're talking about is pretty important

[IPlayKindred]

that is true, this might not even be the problem of the nodes themselves but how the rendering server treats cubemaps in general,

but since it effects reflection probes the most I am hesitant of changing the title of the issue.

I am just not sure if it is, because I cant really pinpoint myself either, but I haven't seen a single mention of cubemap resolution in reflection_probe.h and .cpp

but yet again I am waay out of my depth on this.

[Calinou]

On the other hand, https://github.com/godotengine/godot/issues/64683 says that the reflection size setting is working.

[lyuma]

The reflection size seems to only take effect after reloading the project. Maybe godot keeps using the existing buffers? Perhaps the fact that it requires restart is what this issue is about...

And yes there's a question of memory usage such as #64683 ... If I add 16 reflection probes at 1024, Godot editor uses 6 GB of VRAM.

@IPlayKindred What GPU do you have (how much VRAM does it have)? It would really help if you go to Help -> Copy System Info and paste that here.

If I up the resolution to 2048 then I can definitely chew through that precious VRAM: But at this point we're talking about a different issue.

Personally I'd like to be able to VRAM compress my reflection probes and use different resolutions for some, but Godot's architecture does not currently allow that due to forcing them all into a single uncompressed array. This is kind of a serious issue for using reflection probes, but I think it deserves to be its own issue / proposal.

[IPlayKindred]

restarting godot does not change the resolutions on the reflection probes, changing the probe count, texture array type, ggx samples, or anything under the reflections menu doesnt change the default 256 x 256 resolution of the cubemaps.

tried changing it manually using the renderingserver, still no change.

I have a gtx 1650 super, with 4gbs of vram, a 4th gen intel i5 and 8gb of ddr3 ram.

so I'd say I need to have around 10 reflection probes for it to just brick my gpu, even then I am not using nearly that much anyways, I have tried setting the max cubemap count to 1 (image just showing what the settings name was)

tried different pcs, a laptop, linux mint, windows 11, windows 10. no, still no changes in resoloution and anything around 1024 just kills godot. lost count over how many times I restarted on each attempt.

one laptop was using a ryzen 5 2400 mobile with 8gb of ddr4 ram and another was running an intel i5 5th gen, and some weird gtx card from 2015.

the vram problem can be avoided and good enough results could still be achieved, 512 x 512 is still pretty good, way better than 256.

its just that godot doesn't wanna render any other resolution.

edit: if godot somehow does appear to change the resolution and have the change appear on the probes after the restart, then it must be widely inconsistent , because I had 3 seperate device, 3 seperate OSs. and not once have I seen the reflection probes run anything other than 256 by 256 after multiple restarts on each.

[IPlayKindred]

I have to apologize since I didnt realize I had to set the reflection probe to update once for it to show different resolutions

also setting the maximum cubemap size to 4, allowed me to set the reflection atlas to 2046 without any problems.

How do you check the resolution on the probes? From my looking at the code of the renderer they don't have any resolution themselves, just a size, where do you see that?

I take screenshots of every change and look for jagged "edges" in the reflections

this is what it looks like when its set to update once

its running 2046 res reflections

but why is this only possible for reflection probes that are set to update once, it makes it nearly useless for moving objects like car.

I again apologize heavily for my lack of research. atleast now if anyone has any problems with reflecions probes they'd stumble upon this page.

does this change this from being an issue to a proposal? should I write that proposal? furthermore why is it only possible for update once probes to achieve this?

should I just change the title to specify "update always" reflection probes if its still counts as an issue?

I feel as its technically still an issue since reflection probes losing a large amount of quality due to the update mode is really bad.

again, I am really sorry for my lack of research, sorry for wasting time.

[AThousandShips]

Are they updated when moving the probe?

[IPlayKindred]

yes, once every second, how probes in that mode normally behave. which is jarring for fast moving objects like cars.

[AThousandShips]

Meant the broken state with resolution, does it fix the issue?

[IPlayKindred]

the issue of godot dying is fixed once the Reflection count is set to a low enough value, below 4 for me.

the issue of the resolutions being stuck to 256 is fixed once the update mode is set to UPDATE_ONCE

it stays at the specified resolution (2046 for example) even when the probe is moved.

but setting the problem to UPDATE_ALWAYS sets the resolution of the reflection probe to the default 256 x 256

even when its moved

[clayjohn]

Updating a reflection probe every frame is super slow for two reasons:

1. You are doing 6 render passes to draw the scene for the reflection probe
2. You have to do the full environment map convolution on all six faces of the probe (and for all mip levels)

On mid range hardware this can easily take 20ms or so.

Because of that, we implemented a super fast approach to calculating the environment maps. It takes <1ms on mid range hardware which makes it suitable for realtime updates. However, that approach uses a fixed look up table, so it can only be used with cubemaps with a resolution of 256. We accepted that tradeoff because the speed benefits make anything else not feasible.

In practice, even AAA games limit their reflectionprobes to resolutions of 256 or lower due to performance constraints.

That being said, for Skies we have the same problem, but there we allow users to force using the slower method if they want higher quality. We could probably expose a setting to force using the high quality mode on ReflectionProbes (users will just need to be careful when using it)

[Calinou]

That being said, for Skies we have the same problem, but there we allow users to force using the slower method if they want higher quality. We could probably expose a setting to force using the high quality mode on ReflectionProbes (users will just need to be careful when using it)

For context, see discussion starting from this comment: https://github.com/godotengine/godot/pull/55929#issuecomment-993839196

[Lasuch69]

You are doing 6 render passes to draw the scene for the reflection probe

Wouldn't multiview feature fix this? I understand that it isn't compatible with 100% of devices and is Vulkan only AFAIK, but I think it's worth mentioning.

Edit: I used it in my own renderer on RX 550 2GB for image based lighting.

[Calinou]

Wouldn't multiview feature fix this? I understand that it isn't compatible with 100% of devices and is Vulkan only AFAIK, but I think it's worth mentioning.

I thought multiview was only relevant for mostly matching views, such as XR or stereoscopic 3D in general. Cubemap views use entirely different camera angles, with not much in common being rendered (apart of huge meshes).

[Lasuch69]

I was using multiview to only draw screen sized triangle for pre-filtering, so I don't know how it handles geometry. :/

Edit: I realized that using multiview will most likely result in no culling which means that you would need to render whole scene 6 times, which for high polygon count means exploding GPU.

[IPlayKindred]

even if using higher res reflection probes and updating them every frame would be heavy on performance, the option is still important

512 shouldn't have that much of an impact, and since update_always nodes default to 256 and stick to it, you cant even change it to 128.

In practice, even AAA games limit their reflection probes to resolutions of 256 or lower due to performance constraints.

that is true, but in the car game sphere, that only true for older car games, newer have higher res reflections.

need for speed most wanted from the year 2012, released at the end of the 7th generation of consoles had a 256 by 256 reflection quality by default, (proof by picture above)

but a year later need for speed rivals released for 8th gen hardware and had higher res reflections on them.

look at the difference in sharpness in the cars front windows, the reflection of the tree is much sharper on rivals, the game on the right

rivals is an 11 year old game, yeah sure most triple A games stick to 256, but racing games really dont ever since over a decade.

worst case scenario I manually set the update mode to update_once for a few frames then back to update_always for frame than back again to update_once for a couple of more frames.

that way the reflection probe updates every like 2 frames if I want to.

but even changing the update mode on runtime just switched the resolution to 256, so I cant really do that either

and yeah I'd admit 1024 is really demanding, and 2046 is insane, but what about future proofing? what about high end hardware?

I can set the reflection atlas size and ask the player to restart the game, so I can add the graphical option if someone CAN run it.

the performance point is correct to an extent until we talk about games that rely on it, and the option is still very important.

I do fully understand that godot is the furthest away from a car game oriented game engine, but I am only asking for a hard coded cap to be removed and replaced with a warning instead.

if someone is knowledgeable enough to know the reflection probe node even exists, they can realize the performance impact.

IMPORTANT NOTICE: I can't find concrete numbers for any racing game regarding reflections except for most wanted 2012 because of a config mod and assetto corsa (a game from 2014) which gives you the option of reflections resolutions anywhere from 256 to 2046. but I am sure rivals (a game from 2013) defaults to higher values than 256 on the ps4 and xbox one

[SpockBauru]

Indeed be able do lower the resolution bellow 256 is something that would be beneficial, specially with complex textures that hides the low resolution. It also enable to make a low setting so players can choose depending on their hardware. A player with integrated graphics can have a low setting with 128 while a player with an RTX 3070 can chose a "high" setting with 512 and the rich boy with an 4090 can use the "placebo" setting with 1024.

[clayjohn]

@IPlayKindred Is that resolution setting in need for speed the resolution of a cubemap face, or the resolution of the whole texture? I expect that a game like need for speed would use something like dual paraboloid textures. A dual paraboloid texture with a resolution of 1024 is the same resolution as a cubemap texture with 256x256.

Have you tried setting the update mode to ONCE to see what kind of performance you get? The reflection probe updates every frame that it moves (even with update mode ONCE). So if you move the reflection probe every frame, you will have an idea of what performance you can expect if we add an option to use the high quality version with UPDATE_ALWAYS

[IPlayKindred]

Is that resolution setting in need for speed the resolution of a cubemap face, or the resolution of the whole texture? I expect that a game like need for speed would use something like dual paraboloid textures. A dual paraboloid texture with a resolution of 1024 is the same resolution as a cubemap texture with 256x256.

most wanted 2012, was a game made by criterion, the people behind the renderware engine, which did use dual paraboloid reflections, same way gran turismo 4 and other racing games handled reflections at the time (2001 to 2008).

but since 2008, renderware had a major overhaul since the influx of support by ea, and it rebranded to the chameleon engine now closed from the public and is a couple of hard drives away from being lost media.

by 2010 to 2011 it already supported pbr materials which was extremely modern at the time, so it wouldn't be far off to say they used cubemaps.

from 2012 to today nfs now uses frostbite. which also has cubemap reflections.

the cars look immaculate since hot pursuit 2010, no reflection stretching, and they work from every angle perfectly, so I doubt they have been using dual paraboloids.

its really hard to say anything about the reflections used because aside from chameleon engine which was criterions baby child, everything else is buried behind NDAs.

Have you tried setting the update mode to ONCE to see what kind of performance you get? The reflection probe updates every frame that it moves (even with update mode ONCE). So if you move the reflection probe every frame, you will have an idea of what performance you can expect if we add an option to use the high quality version with UPDATE_ALWAYS

I am getting a smooth 60 fps, 1080p using 2046 res reflections on a moving car when its set update_one

on a gtx 1650 super with 4gb of ram.

---

not as important but I wanna add that gran turismo 7 uses 256 textures on the ps4 version and 512 on the ps5 version.

gran turismo 7 reflection reference point

I know it's 512 because this is what it looks like in assetto corsa

note that the use of 256 can be chalked up to gran turismo 7 being a later ps4 title which is cross gen, meaning the ps4 would be handling alot more lighting quirks and features that earlier titles like rivals didnt have to deal with.

meaning nfs rivals did do something like 512 textures because the ps4 didnt have to alot of other stuff alongside it.

[clayjohn]

I am getting a smooth 60 fps, 1080p using 2046 res reflections on a moving car when its set update_one

That is promising! I'd suggest using that for now until we implement more controls on the ReflectionProbe. The downside to this workaround is that the reflection won't update in frames where it doesn't move.

[SpockBauru]

Sorry, but I don't think that incremental updates like in update_once is acceptable for cars in racing games in 2024.

I remember that even 14 years ago I disabled reflections on racing games that updates once several frames this since it's distracting and just feels bad to play. I prefer to pay the performance price and be able to set the resolution of full real-time reflections.

But I agree that issues like #64683 takes priority right now. I can make some workarounds like use render textures and split reflection probes, but #64683 is something that I can't handle at all.

Please let this discussion open, be able to set the resolution of real time reflection probes is indeed important both for more and for less than the imposed resolution, and can be tackled after the memory issue is addressed.

[IPlayKindred]

@clayjohn its less of a control problem and more of removing a cap set by the engine to not change the resolution on update_always nodes

it would fix this issue immediately,

I read from @Calinou thats what stopping the update_always nodes from reaching higher or lower reseloutions is the filter mode, and that splitting both for this issue to fixed is planned.

so I guess this is issue is bound to fix itself sooner or later.

[BastiaanOlij]

The question I have, and I don't know enough about what other engines/racing games do, is whether it's needed to do a full reflection probe update.

When we are looking at reflections on your own car, there is no point in using a reflection cube, you're doing a lot of reflection rendering for reflections you'll never see. It's likely enough to just render an upwards view.

Same for reflections in other cars, there is no need to render all sides of the reflection cube, you're only ever going to see reflections on the sides that face the player camera.

So I think there is a lot of optimisation here that is very use case specific, something that does well in a purpose build racing game renderer but is far harder to properly embed in a generic engine.

That said, it would be good to figure out what sort of options such engines apply and how we could control that here, even if its just a matter of supporting more reflection shapes and settings that limit which reflections are updated.

I do agree that we should look at the limitations we set here, some do seem very artificial. For professional racing sims clients are happy to just throw the most powerful GPUs at it. So I think it's worth to just make the reflection map, even for dynamic reflections, more flexible.

That said, there are other things to consider in the architecture of a generic engine like Godot that are not easily solvable without concessions for other types of games. One I was discussing with @m4rr5 a little while ago is that we recalculate the main directional shadowmap for each viewport and probe, while in a racing game this can be done once as nearly all viewports render something close to the player location.

Slightly of topic but also relevant, both as a point that we can't optimise for a single use case (e.g. racing games) without effecting developers of other types of games.

[Calinou]

The question I have, and I don't know enough about what other engines/racing games do, is whether it's needed to do a full reflection probe update.

Unfortunately, yes, outside of using raytraced reflections (which are almost always slower than rendering a cubemap every frame). The only exception to this are car bonnet views, where using a flipped screen-space texture can suffice.

Remember that in a chase camera or even a cockpit view, you will see your car's metallic surfaces at various angles, including some that are nearly parallel or perpendicular to the camera. This is why a single render wouldn't cut it and a full cubemap is needed.

@m4rr5' suggestion of performing a single shadow pass would certainly improve performance in the case where you want shadows in your cubemap reflections, but a lot of racing games don't enable shadows in cubemap reflections or in mirror views in general.

Also, it's common for racing games to use the same cubemap (centered around your car) for all cars in the scene. It can lead to slightly incorrect reflections on other cars, but you usually won't notice and it looks much better than the alternative, which is to have no reflections at all on opponents' cars.

Some engines have toyed around with the idea of using tetahedral renders as opposed to cubemaps (e.g. for shadow rendering), but it comes with quality compromises and I'm not even sure if it makes sense from a performance standpoint for reflections.

[m4rr5]

Sorry, but I don't think that incremental updates like in update_once is acceptable for cars in racing games in 2024.

I remember that even 14 years ago I disabled reflections on racing games that updates once several frames this since it's distracting and just feels bad to play. I prefer to pay the performance price and be able to set the resolution of full real-time reflections.

@SpockBauru, first, as a disclaimer, my background is in PC racing simulations, so I have a bias towards those.

I think there are a couple of trade-offs here to consider:

1. As @Calinou mentions above, having some kind of reflection at all is often better than having none at all. This is especially true for cars. We used to offer users the option to choose from "static" (only rendered once, so you do see some clouds and environment reflected but it's not moving) all the way up to "dynamic" (rendered each frame in high resolution) with a few steps in between where smaller maps were rendered less often.
2. Certainly for racing simulation on the PC, people often go for high framerates (as you often see in competitive esports) but here it is even more important as at speed, vehicles move a lot each frame, so it's quite normal for people to race at frame rates of 144 Hz or more (preferably on gsync of freesync screens). Now given such framerates, even if you render these probes every other frame, they will still be rendered at 60+ fps so they will look quite smooth.

[m4rr5]

When we are looking at reflections on your own car, there is no point in using a reflection cube, you're doing a lot of reflection rendering for reflections you'll never see. It's likely enough to just render an upwards view.

Same for reflections in other cars, there is no need to render all sides of the reflection cube, you're only ever going to see reflections on the sides that face the player camera.

@BastiaanOlij I have not seen such an optimization before. Thinking about it a bit more, I think this works depending on the actual shape of the body that you see in your view. For a more classic car, where the bonnet is almost flat, you do indeed end up with reflections of mostly the top, front, left and right sides. However, if you look at modern prototype and formula cars, those body shapes are way more complex and I think you'll end up needing all six sides. That said, you could optimize that by configuring what to render based on such shapes, and if you add a little debug view where each side of the cube has a very distinct color, and render just the reflection in the body, you can even quickly see what colors end up on the screen and decide what sides to render.

Thinking out loud (and feel free to tell me why this won't work) for rendering cube maps, you can split the work into 6 different parts (the rendering of each individual side). With that as a starting point you can decide to either render all 6 sides every frame, or split up the work over multiple frames. The latter would introduce small errors where the sides meet as not every side is rendered based on the exact same position. It would mean that you would only have to render 3, 2 or 1 side per frame, which is better for performance. Now you could take this even one step further and decide per side how often it would update. Let's consider a car moving in a straight line at high speed. In this case, what gets rendered in the left, right side needs to be updated quickly, possibly every frame. The front and rear side are not going to change that much (you're basically just zooming in and out there). Similarly the top, looking at the clouds, will not move that much while the bottom (if it's relevant for reflections) will move more quickly as you're close to the ground here.

So really in the ideal case, for each side being able to control the frequency of updates (ranging from "not at all" to "every frame") would be great. And one could probably also argue something similar for the size in pixels of each side (where rendering fluffy clouds in the sky can be a much lower resolution than rendering a guard rail close to the car).

So I think there is a lot of optimisation here that is very use case specific, something that does well in a purpose build racing game renderer but is far harder to properly embed in a generic engine.

That would be my main take away too, please allow game developers to optimize for their use case.

That said, there are other things to consider in the architecture of a generic engine like Godot that are not easily solvable without concessions for other types of games. One I was discussing with @m4rr5 a little while ago is that we recalculate the main directional shadowmap for each viewport and probe, while in a racing game this can be done once as nearly all viewports render something close to the player location.

Yes, @Calinou mentions this above too, racing games typically would re-use the same reflection cube/probe for all cars, simply because it's way too expensive to render 50 cars that way and you barely notice the difference if you're not looking for it. Re-using what you have is not just done for the reflections, you see similar things being done for example for raindrops being rendered on a windshield. If you're looking with an external camera at a field of cars coming at you, then again rendering 50 different dynamic textures (with windshield wipers clearing drops) is just too much. So you end up using one texture for all cars (even if some cars have one wiper and others have two, yes people hardly notice that until they look for it).

[m4rr5]

Remember that in a chase camera or even a cockpit view, you will see your car's metallic surfaces at various angles, including some that are nearly parallel or perpendicular to the camera. This is why a single render wouldn't cut it and a full cubemap is needed.

I agree, but see my comment above, there might be specific optimizations possible depending on the shape of the body and the presence of such metallic surfaces.

To complicate matters a bit more, if you are actually rendering a view inside a closed cockpit of a car, there is another issue. Let's say you have a metal door to the left of you, with the system we're looking at right now, that would reflect the world outside, so the right side of the cubemap. That ends up looking very weird, as there is a door on the right side in between. So if you want to improve that, you need to first decide what parts of the image should reflect an "outside" cube map and what parts an "inside" cube map. And you'd have to render both. Or, simply decide not to reflect anything for the "inside" cube map.

@m4rr5' suggestion of performing a single shadow pass would certainly improve performance in the case where you want shadows in your cubemap reflections, but a lot of racing games don't enable shadows in cubemap reflections or in mirror views in general.

That is correct. Where the "single shadow pass" would really shine (pun intended, sorry, ;) ) is when rendering triple screen views, where three monitors are rendered, the side ones at an angle, and you could re-use the shadow maps between those three views (obviously you'd have to make sure the single shadow map you render covers all three views). Rendering only one instead of three maps will certainly help.

Indeed in most other views current racing simulations do not render shadows, which is not to say it's a bad idea. As performance increases, this becomes more valid, and especially if you can re-use the same shadow map in all those views.

[IPlayKindred]

Thinking out loud (and feel free to tell me why this won't work) for rendering cube maps, you can split the work into 6 different parts (the rendering of each individual side). With that as a starting point you can decide to either render all 6 sides every frame, or split up the work over multiple frames. The latter would introduce small errors where the sides meet as not every side is rendered based on the exact same position. It would mean that you would only have to render 3, 2 or 1 side per frame, which is better for performance. Now you could take this even one step further and decide per side how often it would update. Let's consider a car moving in a straight line at high speed. In this case, what gets rendered in the left, right side needs to be updated quickly, possibly every frame. The front and rear side are not going to change that much (you're basically just zooming in and out there). Similarly the top, looking at the clouds, will not move that much while the bottom (if it's relevant for reflections) will move more quickly as you're close to the ground here.

while most racing games don't do something as advance, and some I cant even confirm if they do at all. assetto corsa however does have a setting exactly for that (not updating which side the camera is looking at most part)

and it can cut the rendering impact to a 1/6 of the original. while beraly looking weird.

The question I have, and I don't know enough about what other engines/racing games do, is whether it's needed to do a full reflection probe update.

they're not that complicated from what I have heard.

car reflections were beraly even a thing until the gamecube and ps2 era. in the ps1 we had games like gran turismo 1 and 2 using a static cubemap, which was swapped out based on the track to better blend in with the environment.

in daytime the reflections were beraly even noticeable.

once we got to the ps2 era we started getting more "half spherical" reflections, the reflection probe in something like gran turismo 3 and 4 would use 3 cameras around the car (front, back and up) to simulate reflections,

https://youtu.be/adXe2VK3qpc?t=84

that's why in the video you can see the reflections, while good at a glance, look very jarring.

when we enter the ps3 and xbox 360 era, we start getting the reflection probes we have today, the same tech we've been using up until gt7

https://youtu.be/yDJqhPgrIZE?t=2273

6 cameras, on each side, updating the reflections very dynamically, ranging from 256 (burnout paradise - 2008) to 512 (gran turismo 7) resolutions.

some games update faces individually instead of all at once, but that's as much depth as it has really.

reflection probes have always been the silver bullet for making a racing game look good, they even handle all the environmental lighting, but even then they've been really simple behind the scenes.

godot just needs to give the devs the options on which face updates and when, and also unlock the reflection atlas to dynamic reflections aswell, and it'll be alright.

then, we can just check which side the player is looking at and update them manually using scripts.

and yeah like you guys said, its always just one reflection prob on the players car and it maps to every car, which would look weird in some cases, but most people never notice.