Add per-instance frustum culling and mesh LOD support to MultiMeshInstance3D

[DriverBunny37]

Describe the project you are working on

Any 3d project with millions of small objects to render.

Describe the problem or limitation you are having in your project

1. The current MultiMeshInstance3D does not support Frustum Culling,so it renders all instances no matter how much instances on screen.

2. It does not support LOD switching on individual instance mesh,the lod only happens on all instances at same time. this is useless if you are using a big patch MultiMeshInstance3D.we need per instance LOD.

3. There should be an option to manually add LOD mesh. as MultiMeshInstance3D is usually used to render millions small objects which need carefully modeling every vertex, the lowest lod mesh is always better with a simple quad mesh which you can not get it with the automatic LOD generator.

All these problems are a huge performance impact to any one try to use it.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

I believe we could get at least 50%-200% performance boost on MultiMeshinstance3D if these features are implemented.

i know we could split it into small patches to avoid some issue i mentioned,but it is far away behind the real Frustum Culling and per instance LOD and manually added LOD mesh.

Here is the scene i'm testing,just want let you guys know that there still are someone struggling to get Godot to render some AAA like things.

In my test scene i can get:

250 fps without rendering MultiMeshInstance3D. 50 fps with a big patch of MultiMeshInstance3D. 100 fps with a few hundreds small patchs of MultiMeshInstance3D (using the plugin "Spatial Gardener"). 130 fps with manually set LOD mesh in those small patch of MultiMeshInstance3D in "Spatial Gardener".

In my experience i can get at least 170-200 fps in Unity3D with those foliage rendering plugins like GPUInstancer,Nature Renederer,Foliage Renderer, Flora ...etc ( sorry for i can't reproduce the exactly same scene in Unity) All those Unity plugins i mentioned are supporting the things i suggested and most of them even with more features like HI-Z occlusion and LOD crossfading and data streaming ...etc.

Godot really need a better multimeshinstance3d for everyone to create plugin based on it.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

1. Add Frustum Culling in MultiMeshInstance3D so people don't have to split it into hundreds small patches.
2. Add per instance LOD,so people don't have to split it into hundreds small patches.
3. Add manually LOD mesh supports,so we can get better performance with better visual quality at same time.

If this enhancement will not be used often, can it be worked around with a few lines of script?

I don't think so. every plugin is built based on it and every plugin is suffering for these performance issues.

Is there a reason why this should be core and not an add-on in the asset library?

This should be the core so everyone can build plugin based on it.

[clayjohn]

Have you tried using individual MeshInstance3Ds instead of MultiMesh? Godot 4.x has an autobatching feature for MeshInstance3Ds that essentially constructs a MultiMesh internally after doing frustum culling and occlusion culling. I think technically it achieves the same thing you are asking for and many users have found that it performs better than using a big MultiMeshInstance.

[DriverBunny37]

Have you tried using individual MeshInstance3Ds instead of MultiMesh? Godot 4.x has an autobatching feature for MeshInstance3Ds that essentially constructs a MultiMesh internally after doing frustum culling and occlusion culling. I think technically it achieves the same thing you are asking for and many users have found that it performs better than using a big MultiMeshInstance.

Can't try it,there isn't any plugin could paint meshinstance nodes as foliages and i'm not a programmer just an artist with shallow coding skill. but i guess if it's an small area like my test scene it will work for sure,but when it comes to larger size area like open world it will be impossible to edit scenes. my test scene is an area smaller than 400m x 200m ,there already are 190k foliage instances rendered by 1700 MultiMehsInstance3Ds and it will be stuck for a few seconds every time when you reorder nodes,delete nodes,rename nodes,save the scene. i'm only using 6 types of small foliage,much less than what i'm usually using in Unity3D.

i guess the MultiMeshInstance3D is one of biggest reason people don't use Godot to make big world game in Godot. you can use mesh or third party plugins as terrains but you can't skip the MultiMeshInstance3D part. but Godot got the most flexible VoxelGI and SDFGI with reasonable performance,it's really a pity.

[clayjohn]

Fair enough, that would absolutely have to be something managed by a plugin. 190K nodes is just way to much for the editor to handle right now.

Is SpatialGardener activiely developed? Maybe the author is willing to try out is individual instances instead of using MultiMeshInstance3D

[DriverBunny37]

Fair enough, that would absolutely have to be something managed by a plugin. 190K nodes is just way to much for the editor to handle right now.

Is SpatialGardener activiely developed? Maybe the author is willing to try out is individual instances instead of using MultiMeshInstance3D

https://github.com/dreadpon/godot_spatial_gardener I don't think they are working on it anymore,the latest time they updated it is 4 months ago. it already have some error reports when i use it in Godot 4.3 stable. also,i think the performance issue will be much worse in the larger world which won't be a good idea to use individual meshinstances as people are gonna need millions nodes.

[DriverBunny37]

Fair enough, that would absolutely have to be something managed by a plugin. 190K nodes is just way to much for the editor to handle right now.

Is SpatialGardener activiely developed? Maybe the author is willing to try out is individual instances instead of using MultiMeshInstance3D

Some new info: i tried to simulate using individual meshinstance nodes. here is the result: 200k same meshinstance nodes with a directional light with shadow. each node in picture contains 10k meshinstance nodes. every meshinstance node's visibility range is set to 100m at end. it runs around 250-300 fps in 1440p fullscreen on an RTX4090. a reasonable fps i have to say.i believe i can make it runs faster with manually added lod mesh.

but...the bad side: Godot really can't handle my scene any more. 10GB ram was consumed by Godot. every time when i try to add a new node,change some values in node,Godot will be frozen in there more than 12 seconds. when you try to save scene or play the scene,it will take more 140 seconds. and finally,Godot was frozen in there forever when i try to open a new scene. i have to force close Godot. it took me more than 10mins to reopen Godot. all of these are based on a simple scene with only 200k same nodes runs on my "not bad" pc : RTX4090 + AMD7900X + 64GBram +10TBssd. i really don't think people could accept it. a better MultiMeshInstance3D with Frustum Culling and per instance lod is really needed for Godot to make AAA graphic games.

[octanejohn]

maybe change from text file to binary? (from tres/tscn to res/scn)

[tetrapod00]

Worth noting that you can also get the rendering features of MeshInstance3Ds (culling, LODs) without using nodes by using the RenderingServer directly. Here's a very rough example script, probably with some details wrong (I think I'm freeing the RIDs wrong?):

extends Node3D

@export var count: int
@export var mesh: Mesh

var rid_array: Array[RID]

func _ready() -> void:
    var mesh_rid = mesh.get_rid()
    rid_array.resize(count)
    var scenario_rid = get_world_3d().scenario
    for n in count:
        var row: int = n / 100
        var col: int = n % 100
        var transform = Transform3D.IDENTITY.translated(Vector3(row * 1.5, 0.0, col * 1.5))
        var rid = RenderingServer.instance_create()
        RenderingServer.instance_set_base(rid, mesh_rid)
        RenderingServer.instance_set_scenario(rid, scenario_rid)
        RenderingServer.instance_set_transform(rid, transform)

        rid_array[n] = rid      

func _exit_tree() -> void:
    for n in count:
        RenderingServer.free_rid(rid_array[n])

It's obviously a lot of boilerplate, and even more if you wanted it to be a tool script so you could make changes in the editor. But it might be worth investigating this method if you're writing plugins or a custom system. Poke around in the RenderingServer API, there are equivalents to pretty much every MeshInstance3D (or VisualInstance3D) property.

Of course, most people don't want to write their own plugin, so a friendlier "MultiMeshInstance3D" that looks something like this could be brought into core. Internally it creates many instances on the RenderingServer, but it still only has a single node. This could also be an opportunity to improve the API, too. As it is, you can't quite set transforms directly on a MultiMesh resource in the inspector, because it uses a packed format for the transforms. This is a node that could be prototyped as a plugin before being considered to add to the engine.

[clayjohn]

Exactly what Tetrapod00 says. The instances should not be nodes, they should be managed through code. Doing that is essentially the exact same as what the proposal suggests.

In my opinion, the best option would be for the spatial gardener to support using individual mesh instances instead of multimesh. But Tetrapod00s idea could work well as well.

[DriverBunny37]

https://github.com/user-attachments/assets/5a0ce8f5-c142-4991-a8ec-0e77652047e2

I figured out the solution to use RenderingSever to replace MultiMeshInstance3D. All these foliages are rendered by just one script with Frustum Culling and Per Instance LOD and Custom LOD Mesh. Big thanks to tetrapod00 's code,really helped me a lot! Thank you everyone who helped me!

If anyone has same problem like me,here are some tips: you can get the transform data and mesh data from MultiMeshinstance3D. use RenderingServer.instance_geometry_set_visibility_range to set the culling distance and use it to switch LOD mesh as well. use RenderingServer.instance_geometry_set_cast_shadows_setting to optimeze the lower lod mesh. all these settings can be set per instance without any performance hit.

about the proposal: i think i will keep it opening as my solution isn't an out of box solution. also i haven't got it running in editor mode only in gameplay mode. also my solution works good only when instance count lower than 300k on my pc. the scene starts slower and slower when you try to increase the instance count. it will take 15mins after i increased the instance count up to 2 million. i know i should add some distance based data streaming system to solve it. but all i asked in this thread are out of box thing in any other engine as i know in Unity,in Unreal,in FlaxEngine,in Unigine...

I really hope MultiMeshInstance3D got all the features out of box.