C# CompositorEffect does not work, and it will stuck the engine when re-opening the project

[hippogyz]

Tested versions

Godot v4.3.rc2.mono.official [3978628c6]

System information

Godot v4.3.rc2.mono - Windows 10.0.22631 - Vulkan (Forward+) - dedicated NVIDIA GeForce RTX 4070 (NVIDIA; 31.0.15.3161) - AMD Ryzen 9 7900X 12-Core Processor (24 Threads)

Issue description

I'm trying to write a compositor effect for rendering a custom texture for my rendering proposal.

However, I failed at making translation the official demo to C# version in the first step.

I put the initialization step into the constructor, and marked the class as [GlobalClass] and [Tool]. But it does not work when I attached it to a Compositor of a WorldEnviroment Node.

More severely, if I saved the scene with a C# compositor effect, the engine will be stuck when the project is re-opened.

Steps to reproduce

In MRP, I've made an available CompositorEffect with GDScript and a C# version translation of it. The custom CompositorEffect will make rendered object completely black.

1. Switch the GDScript version and C# version for the WorldEnvironment's Compositor in the main scene.

2. The C# version should take no effect at all.

3. Save the scene with C# version is attached. Then close the engine.

4. Re-open this project, engine will be stuck. (later manually delete the compositor in .tscn file can re-open this project successfully)

Minimal reproduction project (MRP)

CompositorCSharpMRP.zip

[raulsntos]

The part of your code where you create the pushConstants byte array is wrong:

byte[] pushConstants;
{
    using var ms = new System.IO.MemoryStream();
    using var bw = new System.IO.BinaryWriter(ms);

    bw.Write(size.X);
    bw.Write(size.Y);
    bw.Write(0f);
    bw.Write(0f);

    pushConstants = ms.ToArray();
}

The size variable is a Vector2I which means the X and Y fields are int. So the binary writer is writing int instead of float, which has a different binary representation. Casting them to float should fix it:

bw.Write((float)size.X);
bw.Write((float)size.Y);
bw.Write(0f);
bw.Write(0f);

[hippogyz]

The part of your code where you create the pushConstants byte array is wrong:

byte[] pushConstants;
{
    using var ms = new System.IO.MemoryStream();
    using var bw = new System.IO.BinaryWriter(ms);

    bw.Write(size.X);
    bw.Write(size.Y);
    bw.Write(0f);
    bw.Write(0f);

    pushConstants = ms.ToArray();
}

The size variable is a Vector2I which means the X and Y fields are int. So the binary writer is writing int instead of float, which has a different binary representation. Casting them to float should fix it:

bw.Write((float)size.X);
bw.Write((float)size.Y);
bw.Write(0f);
bw.Write(0f);

Oh...I didn't recognize the question about data type before. Casting them to correct type fixes it indeed (I choose int). Now the C# version compositor exhibits same behaviour as the GDScript version in game and editor view.

But the fixed C# version compositor still stuck the engine when I re-open the saved project. Maybe I should modify the information of this issue?

1. Assign the C# version compositor to WorldEnvironment.
2. Save the scene, and close engine.
3. Re-open this project, the engine still will be stuck.

Using GDScript version won't trigger this problem.

Here's the fixed MRP. CompositorCSharpMRP_FixType.zip

[raulsntos]

I'm unable to reproduce that issue. Try opening the editor from a terminal with the --verbose option and see if the output shows any errors or warnings.

[hippogyz]

Once the editor freezes soon after launching, I attach Windows Debugger onto it to investigate. I think maybe there is kind of dead lock in the main thread, here are stack traces for main/worker threads.

By the way, I've taken some experiments on my desktop workstation(AMD Ryzen 9 7900X) and low profile laptop(Intel i7-14650HX).

	Desktop	Laptop
with verbose option	frozen	fine
without verbose option	frozen	frozen

And in the laptop --verbose mode (which is fine), I found it logs three 'Generated 'ID:-XXXXXXXXXXX' preview in XX usec' info. But in the desktop case, there only logs two and editor is frozen. For example:

Loaded system CA certificates EditorSettings: Save OK! Using present mode: Enabled Generated 'res://custom_compositor_effect.glsl' preview in 190 usec Generated 'ID:-9223370289004350081' preview in 11 usec Generated 'ID:-9223370288987572864' preview in 9 usec

[hippogyz]

The freeze question still exists in the 4.3 release version.