Implement mesh splitting for better frustum culling effectiveness and displaying more lights simultaneously

[Calinou]

After a discussion with someone on the #godotengine IRC, it seems Qodot doesn't support mesh splitting (or we haven't found it, at least). Mesh splitting allows for greater frustum culling effectiveness, on top of being able to render more than 8 Omni/SpotLghts for the whole level.

Is there such an option in Qodot?

[Shfty]

Qodot splits brush geometry into Mesh resources by entity, and then into surfaces by texture in order to optimize draw calls.

Basic structural brushes (i.e. ones without a classname) are automatically assigned to the worldspawn, which is a singleton 'static mesh' entity common to every Quake-format map. In order to split brushes out of that while retaining the same 'static geometry' semantics, you can apply the group classname that ships inside Qodot's example FGD (or use func_group in a Quake-based FGD). This will cause them to be treated as a distinct entity with its own Mesh / MeshInstance.

TrenchBroom layers also use func_group entities under the hood, so these can be used to achieve a similar result provided that the Use TrenchBroom Group Hierarchy setting is disabled on QodotMap. Having it enabled will also allow brushes to be split into their own entities, but imposes some additional structural rules that are outside the scope of basic geometry work. (More information available here)

[Shfty]

As for a more automatic workflow, I think that would be best implemented in the form of official BSP/QBSP support since it covers mesh optimization along with various other features.

[Pio64]

Hi, thanks for the explanation. In TrenchBroom I've changed worldspawn to func_group, but I'm still getting a single mesh. What else do I need to do?

[Shfty]

@Pio64 You shouldn't be changing the class name of your worldspawn via TB's property matrix - that'll cause the entire map to spawn as the new classname, rather than the desired behavior of splitting out a specific set of geometry into a separate entity.

The proper way to create a new entity from some existing worldspawn geometry (while leaving the rest intact) is to select a set of brushes, then right click > Create Brush Entity > pick classname.

[Pio64]

That worked, but can Qodot just split everything for me? I don't need any groups, because even without them Godot will barely render any lights in my scene. So I need everything to be separate. Doing those steps manually for hundreds of objects takes a lot of time and I can't imagine doing it on bigger levels.

The separate meshes don't have "Use in baked lighting" flag enabled. Is there a way to fix that without doing it manually?

[Calinou]

That worked, but can Qodot just split everything for me?

If Qodot split every brush into its own mesh, you'd end up with thousands of draw calls quickly. This would greatly increase CPU usage.

[Pio64]

In the level I'm currently working on it would be around 1000 meshes. I have to do this manually anyway to be able to use lights. The amount of lights I can now use is still not enough, but it's a little better than before.

[Shfty]

1000 meshes is a lot for the renderer to deal with individually- I wouldn't suggest splitting your geo with that level of granularity. The recommended approach at the moment is to split your map into room-sized sectors, so each one is a single entity that will build as a single mesh and individually obey the 8 light limit.

(As an aside, splitting by room also lends itself well to implementing sectorized visibility culling for larger maps, since each room is already a scriptable entity that can be shown or hidden via game logic.)

If you need more than 8 lights per sector, you should consider baking them into a GI probe to reduce load on the lighting system. If that's not an option - i.e. because they need to update at runtime, or because your map is too big for a GIProbe to reasonably handle - then you should make absolutely sure that they need to be full-fat dynamic light nodes before you consider further splitting your geo as a workaround.

A lot of many-dynamic-light cases can be replaced with additively-blended shader effects - for example, I had a user who encountered the lighting limit after creating an enemy that dropped an explosion of small pickups on death, each of which had a small dynamic light attached to it. In that case, replacing the lights with an additive billboard glow produced a similar-enough visual result and allowed them to sidestep the limit, rather than introducing a major performance knock-on to the rest of their map through splitting.

If that's still not an option, then manual splitting or some kind of automated tool script solution are the remaining choices. In theory you could pull the meshes apart Godot-side and create one MeshInstance per texture surface (since those are easily accessible via the built Mesh resources), or even manually chunk your level into an ordered grid by slicing the Mesh resources into segments using SurfaceTool or manual ArrayMesh manipulation. That's getting fairly advanced though, and would depend on the needs of your project.

I experimented with per-brush / per-face splitting options in early versions of Qodot, but they ended up being untenable for anything larger than a demo map. Alongside the rendering implications, there were also issues with loading, saving and runtime performance in the editor due to the sheer amount of PhysicsBody, CollisionObject and MeshInstance nodes that end up in the scene tree. It ended up bloating the geometry processing and making maintenance difficult, so the feature was dropped in favor of the current solution, which hands control over to the user through the map editor and better resembles how Quake manages its entities.

It may seem fiddly, but that's the nature of working within the graphical limitations of 3.x - the dynamic light limit will be increased significantly by the clustered forward renderer coming in Godot 4.0, but until then it seems better to require that users understand those limitations and manually opt into a draw call bottleneck, rather than being able to flick an easy 'fix the lighting' switch which will cause one anyway but not make it obvious as to why.

[Pio64]

8 lights per room is simply not enough in a game. I have a small level, but it doesn't get much light from the outside (skybox and directional light). I imagine making a completely indoor level must be impossible.

I am using GIProbes, but they only bake indirect light. I tried increasing propagation and intensity of indirect lights, but without direct light and hard shadows it looks bad. I don't need dynamic lights in my scene at all, but there is no other way to light a scene in Godot. There is no baked direct lighting.

I use some materials with emission. The problem are static lights and making the scene not dim.

The biggest room in my scene has around 20 lights and even after removing some of them and manually splitting every mesh, I still have lights disappearing when camera moves. So it looks like there is no way to solve this problem. Another issue is that most people don't know those things, because nobody really talks about it, so the only way to find out is to waste time trying and fail. None of it is Qodot's fault though. Godot is just unusable for 3D games. I'm glad version 4 will (maybe) allow us to use lights, but after a few years I'm tired of waiting for 3D to be fixed. Thanks for your help and advice :).

[Shfty]

The light flickering you're seeing despite the manual splitting is probably down to how the renderer determines which dynamic lights affect a given mesh - if it isn't sorting them to make sure the most relevant ones are used, then the end result will be the same. In theory a sectorized culling system should be able to work around that to some degree provided that lights are culled alongside the sectors they reside in, but baking is the more practical way to use more than 8 lights as-is.

And while not much by modern standards, a single-digit dynamic light limit was considered the norm for a lot of 3D games before the advent of deferred rendering. If you take Quake as an example, the only dynamic lights are tied to projectiles and certain particle effects due to this; everything else is baked by the BSP pipeline, which is what allows E1M1 to have 16+ lights in its first room alone.

Though there are some caveats, you can actually bake direct lighting with both the BakedLightmap and GIProbe nodes by setting your lights' Bake Mode to All. There are demo maps in res://addons/qodot/example_scenes/0-visuals/5-baked-lighting/ with working examples for each.

The caveats are:

• GIProbe is preferred over BakedLightmap until the new lightmapper is backported to 3.x (or until 4.0 releases)
• GIProbe has a resolution limit that may not be sufficient for sharp lighting across a large scene
• There are some issues with using multiple bake volumes in a single scene, so this can't be used to work around lightmapper crashes or GI resolution limits for large scenes

On the topic of emission, MeshInstances with an emissive SpatialMaterial can also contribute to GI with the appropriate settings, which can be a useful alternative to light nodes in certain cases.

[TheRektafire]

@Shfty I tries using BakedLightmap once but with a map exported from trenchbroom as obj then imported to godot as gltf from blender, unfortunately even in my simple map (a small outdoor area with a small room nearby and a directional light acting as the "sun" the lighting was very leaky and looked kind of bad, like some light from outside was leaking inside the bottom part of the walls of the room :( That could have been because of the way I made the geometry though, idk how picky godots lightmapper is regarding topology(ie whether they need to have properly connecting/overlapping vertices or if they can just be laid on top of each other without being properly connected)

[Pio64]

I point out specific problems Godot has and give examples why it's unsuitable for 3D games. You ignore most of what I said and start talking about something unrelated. I get that you like Godot, but let's be honest here. There is no baked direct light in this engine - something that even Quake had. For this reason you can't replicate Quake maps nor make 3D games with indoor scenes. Godot should not be advertised as a capable 3D engine, until this is fixed. Otherwise it's wasting people's time and those people might be less likely to come back when the issue gets solved.

Lightmapper in Godot (using your own example scene):

After adding a GIProbe:

Even your own readme says it's broken:

@TheRektafire don't waste your time with lightmapper. It's broken and they know it. There is no way to make indoor scenes in Godot, unless you are fine with having no light like in the tps demo.

[Calinou]

Godot should not be advertised as a capable 3D engine, until this is fixed.

The showcase thinks otherwise. Many 3D projects are successfully being put in production.

Also, the lightmapper will be heavily improved by https://github.com/godotengine/godot/pull/40035 once it's merged. We'll try to get it in 3.2.4, but we can't make a guarantee. In the meantime, make sure to use the RayTrace mode instead of ConeTrace in the current lightmapper.

Additionally, when using GIProbe, you should really try to maximize the use of emissive materials for lighting (unless you need real-time shadows). It's more physically accurate to do so :slightly_smiling_face:

[Pio64]

Funny. I watched the first video and there was only one 3D game with an indoor scene (02:43) and the lighting was very dim just like in the TPS demo. I wonder why...

RayTrace mode is broken too.

I use emissive materials, TPS demo does too and they doesn't solve the problem.

[Shfty]

@TheRektafire I don't think topology matters - I've not been able to get workable results out of it for various scenes, hence the recommendation to avoid it in favor of GI until the new version makes its way out.

[Shfty]

@Pio64 As for your issue...

I point out specific problems Godot has and give examples why it's unsuitable for 3D games.

My role here is to provide support for Qodot, not fixes for fundamental engine issues. If the workarounds I suggest for said issues aren't satisfactory, then you should take your grievances up with the Godot team in a more appropriate discussion space than my git repo.

You ignore most of what I said and start talking about something unrelated.

Don't make provably false blanket statements like "8 [dynamic] lights per room is simply not enough in a game" if you don't want them to be picked up on and deconstructed using a real-world example.

I get that you like Godot, but let's be honest here.

I've worked with Godot for a year, most of which has been in a professional context, and thus am very much aware of its issues. Implying that I'm defending it because I like it, rather than simply sharing knowledge from practical experience, is a mind projection fallacy.

There is no baked direct light in this engine - something that even Quake had. For this reason you can't replicate Quake maps nor make 3D games with indoor scenes.

"The lightmapper isn't good enough" is a far cry from "There is no baked direct light", much as "You can't make indoor scenes" is not the same as "I can't light an indoor scene in the exact manner I expected to". Rein in your hyperbole.

Though it is true that you can't replicate the lighting of a Quake map 1:1 using Godot's out-of-the-box baked / dynamic lighting systems. A BSP > OBJ conversion tool would be a better fit for that particular case.

Godot should not be advertised as a capable 3D engine, until this is fixed. Otherwise it's wasting people's time and those people might be less likely to come back when the issue gets solved.

How the Godot team conduct their advertising and user retention operations is nothing to do with me.

And - I'll be candid here - you're the one who chose to use Godot. If you're not ready to deal with the R&D, workarounds and compromise necessary to get solid results, then any frustration and eventual disenfranchisement you experience as a result is of your own making.

Nobody has a gun to your head forcing you to keep using it, and you probably won't be missed if you're already beyond the point of posting passive-aggressive rants on some unfortunate schmoe's git tracker. Put that energy into finding a toolset that works for you instead.

Lightmapper in Godot (using your own example scene):

As mentioned, the lightmapper is not recommended for usage, and that scene exists for when it gets fixed. Use GI instead.

After adding a GIProbe:

There's already a working example scene specifically for GI in the same folder that demonstrates baked direct lighting. Adding your own to the lightmapper scene is just going to cause confusion.

Even your own readme says it's broken

This is not the gotcha you seem to think it is. Every mention of the lightmapper on this page has been disclaimed with "GI is preferred". Use GI instead.

I'll be closing and locking this thread now, since further discussion is unlikely to be productive.