Godot 4.1 Screen flickering in random colors.

[NancokPS2]

Godot version

v4.1.stable.official [970459615]

System information

Windows 10 x64 - Godot v4.1 - OpenGL (Compatibility) - dedicated. GPU: FirePro v4900. CPU: AMD Phenom II X6

Issue description

The viewport is constantly flickering between colors no matter the scene or project. Compatibility mode both in the editor and when running.
The issue does not seem to come into effect during runtime.

Steps to reproduce

• Create project, see the issue.

Minimal reproduction project

N/A

Update:
It seems to be an issue with cached shaders from a previous version(?)
No idea honestly, it seems to be working fine so far.

[Calinou]

• This is likely due to the same cause as https://github.com/godotengine/godot/issues/78936.

If you can compile the engine from source, you could look into bisecting the regression to greatly speed up troubleshooting. In fact, this is probably the only way this issue could be resolved, as nobody else has been able to reproduce it so far.

[NancokPS2]

Thanks, i will try that.

I'll be keeping note of the possible culprits in this comment:
Broken between versions:
v4.1.rc1.official [1f9e540f1]
and v4.1.rc2.official [46424488e]

Working commits so far:
1f9e540f1 <-- Apparently this is bad too 1f9e540f14
9a09ee5d0
f228fb715
16dd4e572 Bad commits:
ab430b0b4
818cee618

Other possible factors:
.godot/shader_cache/

[NancokPS2]

Possible fix found:
Project Setting "rendering/textures/default_filters/use_nearest_mipmap_filter" must be true to avoid the bug.
Probably wrong, ignore.

If a .godot/shader_cache/ folder from a previous versions is used, this may not be necessary (for some reason).

Update:
Looks like it is an issue with shaders not caching properly, deleting shader_cache and re-opening the project seems to fix it. (?)

Re-Update:
The bug tends to re appear when new meshes are created in the editor, but to a less extreme extent. Reloading the project fixes it quickly

[Foxysen]

The effect on my laptop is more like disappearing meshes/graphic, but the causes seem to be similar. Doesn't happen in 4.0.3, sure does in 4.1.0. Appears to pop when it tries to save shader cache. Like it once had graphic glitch out at the same moment as it informed me that it failed to save shader cache but file was locked. But it sorta sometimes happens and sometimes doesn't on successful cache saves.

OpenGL mode. Only 3D scenes.

[Foxysen]

Oh, right, I just noticed the setting to disable shader caching in 4.1.0 (Rendering->Shader Compiler->Enabled)

That fixed it for me, disabling it. NancokPS2 should try it too to pinpoint issue.

I do however get "_save_to_cache: Condition "f.is_null()" is true. <C++ Source> drivers/gles3/shader_gles3.cpp:609 @ _save_to_cache()" error spammed in debugger when it's disabled.

EDIT: Actually, no, that didn't fix it, it keeps doing it at times, just a bit later.

Sure keeps happening when I toogle preview light and preview environment. Those even freeze-before-glitching for a small period of time unlike 4.0.3.

[Foxysen]

I could try to bisect this issue somewhere between 4.0.0 to 4.1.0. But it takes me 6 hours to compile Godot source code from scratch. And on additional compiles from that base I ended up having errors in Godot, ie bad compile, no idea why.

So I would need a compile buddy.

(Have AMD GPU too like these two people, Windows, will write more later)

[bitsawer]

Also possibly related issue where OpenGL rendering worked in Godot 4.0 and then stopped working in 4.1:

• https://github.com/godotengine/godot/issues/78888

This one also seems to hint at shader caching issues, although there seemed to be more problems behind it. The drivers and GPU were pretty ancient. They also did some good release bisecting, possibly pointing to some OpenGL change in 4.1 dev2: https://github.com/godotengine/godot/issues/78888#issuecomment-1615249154 However, this is still just speculation, these issues might have nothing to do with each other.

[Foxysen]

I tested and can confirm that my own bug starts to happen to me at 4.1 dev2 too. 4.1 dev1 is fine.

Also, here is my GPU stats as seen by Godot: OpenGL API 3.3.13399 Core Profile Forward-Compatible Context 15.201.1101.0 - Compatibility - Using Device: ATI Technologies Inc. - AMD Radeon(TM) R5 M230 My low-tier laptop actually comes with dual AMD R4 Graphics and R5 M230 GPU thingie.

Mine, I think, are related with #78936 too.

[bitsawer]

A lot seems to point to https://github.com/godotengine/godot/pull/76092. It might be correctly implemented but old or buggy GPU drivers might just trip on that feature somehow, the general OpenGL driver quality has never been too high.

If anyone wants to test, they can try to build Godot with that one-line patch by ChibiDenDen: https://github.com/godotengine/godot/issues/78888#issuecomment-1616436698

Another alternative is to simply patch ShaderGLES3::_load_from_cache() and ShaderGLES3::_save_to_cache() in shader_gles3.cpp to immediately return without doing anything and see if that helps.

[Foxysen]

Ok, did some compiling.

ChibiDenDen's one-line patch did nothing, bug still present.

Then I played with making ShaderGLES3::_load_from_cache() and ShaderGLES3::_save_to_cache() functions return early: 1) Neither return early - bug present 2) Only load returns early - bug present 3) Only saving returns early - fixed 4) Both return early - fixed

So it must be something about saving to the cache (that doesn't get disabled by Rendering->Shader Compiler->Enabled option?).

I did a bit of early return/break testing within cache saving function and I notice that my rendering starts breaking after glGetProgramiv(specialization->id, GL_PROGRAM_BINARY_LENGTH, &program_size); line. Breaking/returning earlier has no bug. I don't know if later lines may cause the bug as I didn't shuffle, just inserted break here and there.

I know nothing about rendering, so I would need clear instructions if you want to test further.

[bitsawer]

Thanks @Foxysen, you have been a great help so far, looks like we are slowly closing in on a solution or at least a workaround here. None of the team members here seem to have access to hardware or drivers causing this issue, so debugging it is pretty hard.

So it must be something about saving to the cache (that doesn't get disabled by Rendering->Shader Compiler->Enabled option?).

It seems like both GLES3 and Vulkan shader caches are always enabled in editor mode. Only when playing/running the project or an exported version does the setting take effect. So it makes sense why forcefully patching it out helps:

https://github.com/godotengine/godot/blob/6588a4a29af1621086feac0117d5d4d37af957fd/drivers/gles3/rasterizer_gles3.cpp#L278-L281

In theory, you might be able to bypass shader cache and this bug by running the project directly on command line, bypassing the project manager and editor. Adding this:

[rendering]

shader_compiler/shader_cache/enabled=false

to your project.godot file by using a normal text editor (but don't add duplicate [rendering] sections if there is already one) and then running the project directly without going through the editor like this (on Windows):

godot_someversion.exe --verbose --path C:/Projects/YouGodotProjectDirectory > godotlog.txt 2>&1

should in theory bypass shader cache and this bug, but I haven't tested it. Important: use --verbose! It should also try to log more OpenGL errors. This will redirect output to godotlog.txt file in your current working directory.

The 2>&1 is there to redirect both normal output and error output. Run Godot until you encounter the problem or crash. You can then attach this text file to your Github post while editing or writing it by simply drag-and-dropping it. The argument after --path should be your Godot project directory, it's where the project.godot file is located. You can see some other possibly useful arguments in https://docs.godotengine.org/en/stable/tutorials/editor/command_line_tutorial.html

I'm a bit busy this weekend, but I'll try to put a special debug version build/PR out early next week with some extra logging and error handling, I'll report back here then. It might show something useful and makes it possible for other people to test things, too.

Everyone else is free to report their experiences here, too. In fact, that could be extremely useful, so no need to hesitate. But don't forget to post your computer and GPU specs.

[Foxysen]

Getting to disable cache in editor would have been a good workaround for us old hardware havers.

Hm, I am trying to understand what you want me to do. You want us to disable shader cache and then run Godot on non-exported project, bypassing the editor, while logging everything? Wouldn't it be useless since we won't encounter shader cache driver bug due to it being disabled in non-editor? Just making sure that it's related to shader cache this way?

[bitsawer]

Now that I think of it, you could do both. First, just run the project editor normally in a stock Godot editor using command line, but redirect the output to a file (note the added --editor argument here):

godot_someversion.exe --verbose --editor --path C:/Projects/YouGodotProjectDirectory > godoteditor.txt 2>&1

Then you can do the project.godot editing trick and run the project directly, bypassing the editor (without the --editor argument)

godot_someversion.exe --verbose --path C:/Projects/YouGodotProjectDirectory > godotplay.txt 2>&1

If you can add both logs (godoteditor.txt and godotplay.txt) to your post here, that could be useful. Note that there can be some directory names like your user/home folder in the logs, you can replace them with YYYY or something if you want to.

[Foxysen]

godoteditor.txt godotplay.txt

Ok, played around, including looking at verbose console while doing things, so I have a good idea what error messages it prints when bug pops up... None! Still posting logs anyway tho.

For editor log file (shaders are enabled!), I played around until I made the bug appear. From what my experience with console, my actual editing session started after WARNING: Blend file import is enabled in the project settings, but no Blender path is configured in the editor settings. Blend files will not be imported. at: _editor_init (modules/gltf/register_types.cpp:73) And then I played around, made bug come out, and the next line ERROR: 1 RID allocations of type 'N5GLES37TextureE' were leaked at exit. was printed only on editor exit, I believe.

For editor-less version, I actually had hard time making the bug appear even with shaders enabled, at least in my test project that doesn't push complex things. Just once and I missed it. None the less, the editor-less log is shader-less, just as asked, by editing project file. The bunch of errors at the end have always appeared to me, even in editor, when shaders are disabled, with no direct relation to bug.

I have run a normal stable 4.1 Godot.

[clayjohn]

@Foxysen Could you test https://github.com/godotengine/godot/pull/79883?

While taking a look at the shader caching code, I noticed that we never check if the cache location is available. But the check is required as we use it as a shorthand to check if the shader cache is even enabled.

This might be enough to fix your issue. However, it may only be enough to fix the fact that you can't disable the shader cache. I would appreciate testing from someone who can reproduce the issue

[Foxysen]

Ok, tried it. It sure doesn't help me in Editor mode, but to my knowledge you can't disable it in Editor, which is where my biggest hurdle is. I had hard problem making bug manifest in my exported project to begin with, so I can't really verify it. Hmmm....

[bitsawer]

Took a bit longer to dig more into this, but I made a draft PR that might help or at least provide more logging info: https://github.com/godotengine/godot/pull/80680 Anyone who is affected by this issue is free to either build their own Godot using that PR or download a build artifact and test it. If you want to build Godot yourself, either fetch the PR using git or apply a patch version: https://patch-diff.githubusercontent.com/raw/godotengine/godot/pull/80680.patch

If you can't build Godot yourself, you can download a build by logging in to Github and downloading an Godot editor build artifact corresponding to your operating system, they should be available in the Checks section https://github.com/godotengine/godot/pull/80680/checks, you should download the editor build, for example windows-editor:

@Foxysen, can you test the PR either way and see if it helps or not? The PR also adds a lot more logging, so posting the log file here might reveal something more useful this time. As usual:

godot_someversion.exe --verbose --editor --path C:/Projects/YouGodotProjectDirectory > godoteditor.txt 2>&1

Everyone else if also free to report any useful information.

[Foxysen]

Downloaded artifact for Windows 64-bit editor.

Didn't help.

godot_foxysen_log.txt Here is a log however. For some reason those launch options just put info into in-editor console but it's fine, I copy-pasted everything I had while running project editor. I stopped and closed after the last "_save_to_cache() start:" because that was the one that made the glitch appear.

(I am close to just running 3D Godot projects on Linux where I don't have any OpenGL driver bugs, but have like less FPS, probably picked too heavy Linux OS with Gnome, heh ow. But still going to assist on this rather annoying bug. If you can't figure it out, just add launch flag to disable shader cache for editor, put FAQ, I already have to run "--single-window" on Windows.)

[bitsawer]

Thanks for testing @Foxysen. I just updated the PR with a new version with some small OpenGL changes (like using an OpenGL shader cache hint when creating shaders) and logging tweaks. If you can also test that new version and post the log, I would appreciate it. I doubt it will make much difference, but a there is always a chance.

I also added a command line switch (--disable-shader-cache) to disable OpenGL shader cache in that PR, so after testing the new version like previously, you can now also run Godot like this:

godot_someversion.exe --disable-shader-cache --verbose --editor --path C:/Projects/YouGodotProjectDirectory > godoteditor.txt 2>&1

That command should hopefully work better as it should fully disable shader caching (but it does not forward that argument yet to the child process if you press play in editor). No need to post log for this one if it works. We might have to add a manual switch like this anyway, but it would be better if we could detect the situation programmatically if possible so that Godot users can target as many systems as possible without having to force disable cache by default for everyone. Unfortunately, it looks like some drivers are just too broken unless we want to manually create some kind of blacklist.

[Foxysen]

Downloaded artifact for Windows 64-bit editor.

It seemed to take a while to make the glitch manifest but it still did for me. godot_foxysen_log3.txt In-editor log with glitch above. I played around in 3D test scene until I created new scene and went into editor to enable rendering/shader_compiler/shader_cache/enabled and created mesh, light, and had it appear at the end of the log.

Frankly, for some reason, I couldn't yet make glitch manifest with rendering/shader_compiler/shader_cache/enabled disabled despite it writing to the cache in console. I think it's just dumb luck in 30 minutes I've spent around and I am not sure if it has any impact, as it's not supposed to do anything in the editor. But who knows, I didn't follow 4.2 Godot main repo, so maybe there is something there now. ...Then again, I think I had half of gizmo disappear even then. Bleh. So maybe just luck.

The launch switch --disable-shader-cache works however in the editor, yap, no complaints.

[Foxysen]

But yes, I welcome other people with the shader cache impacted OpenGL driver bugs to check this artifact out. I think I might have exhausted the best out of my side. Maybe others will get a more specific error/log behavior.