Improve Godot Engine's boolean operator engine (CSG) with `elalish/manifold`

[fire]

• [x] Restore normals and tangents
• [x] Postpone running a profiler and see if there's easy bottlenecks
• [x] Write a Godot Engine proposal
• [x] Generating CSG nodes is slow on a full character mesh added to a box
• [x] Can start providing test cases that crash the godot engine patched with manifold
• [x] Remove exceptions from elalish/manifold ?
• [x] Propose idea
• [x] Ask for comments
• [x] Investigate code details
• [x] Tech lead of Godot Engine is ok with requiring manifold geometry inputs for CSG
• [x] Remove requiring Cuda Developer Kit to build.
• [x] Check performance latency of mesh generation (per frame? per second? per minute?)
• [x] Blocked on removal of boost dependency
• [x] The results of boolean operations look great. No gaps.
• [x] Restore UV coordinates
• [x] Restore materials

[fire]

I upgraded the manifold library https://github.com/V-Sekai/godot/tree/csg-manifold-06.

[pca006132]

FYI: #111 can probably help you get this working with cmake on Windows

[fire]

Godot Engine doesn't use cmake, but yes, that was a required fix in my manifold fork.

[fire]

@elalish What's the best way to check if the input meshes I have failed and have the test cases be recreatable on standard manifold.

I think the is manifold or the operators are faulty.

[elalish]

You're welcome to create a test that demonstrates the error you see. Are you getting a crash or an exception?

[fire]

The newer master improved the crashes.

However, I had great difficulty getting the git sub-modules to be easy to use.

[pca006132]

What's the problem with the submodules? Issue with the build script? If you don't need to run the manifold tests on your repository and don't need model import/export (e.g. when provided by another library), you can ignore the google_test and assimp submodule which should be the most complicated to build. Thrust and glm are header only and should not be hard to deal with.

[fire]

Thrust was difficult to build. They use a structure where the dependency/cub is linked to the cub (a text file with a path) in the thrust/cub.

Thrust also has cub and a cuda library submodules.

[pca006132]

I see. Wonder if it is possible to build it cloning the cub and libcuda submodules when cuda is disabled. In principle you don't need that if you don't use CUDA.

[fire]

I was a bit tired and approached the thrust library wrong.

1. I have to fork thrust
2. Change fork to use a cub dependency that points online and not to a ../cub link
3. Use this in manifold.

I was able to get this into godot with submodules. It'll be an easier task to remove submodules later.

[fire]

@elalish What's the best way to provide the test cases?

Most of the crashes that create false exceptions are done when I'm moving the merge / union / intersect. How to encode this? I can do videos and provide test scenes and an executable for whatever os you want. Godot is a single executable.

[pca006132]

I guess the problem you are facing is #148, we can already reproduce it and just need some time in fixing it.

[fire]

I have updated the godot engine pull request with the overlap changes.

[fire]

Screencast from 2022-07-22 11:01:47 AM.webm

@pca006132 @elalish Does this look like anything obvious?

Empty means is_manifold failed.

[pca006132]

No, this does not look abvious (at least to me). Can you export these models so we can add it to our tests?

[elalish]

Agreed, it would be great if you (@fire) could create a debug function to dump the inputs when the output is bad so we can repro. Also, what is empty? Is that when it's blue? What is the difference between the cube that stays, the cube that vanishes, and the head that turns blue? It's a bit hard to tell what's going on, since the Boolean only takes two input meshes.

[fire]

I'm not that familiar with the inputs. What are the input formats for the tests?

How do you prefer them?

[pca006132]

I think the usual 3D model format is fine, stl/glb etc. And please provide us with the coordinates in the scene so we can reproduce the exact coordinates.

[fire]

After the changes, I can't seem to recreate the posted errors. :tada:

Edited:

New crashes. They crash inside of manifold, but from invalid data!? I'll change the godot type from primtives to meshes. Trying clang asan and bsan.

[Frontrider]

It's just something I'm going to throw in as an idea that might help with runtime performance. Could this feature be improved by having an option to not export the CSG nodes themselves to the exported game, but say "replace the combiner with a meshinstance". Not actually replace because that could break scripts, but have it display a static mesh that was created in the editor? (with warnings that you should NOT reference child nodes if this option was selected)

IF CSG is already meant for static geometry due to speed then this could make it a more viable production tool. I see that it's meant for prototyping and not released games, but a small studio or a lone dev can still make really good use of it.

[Calinou]

Could this feature be improved by having an option to not export the CSG nodes themselves to the exported game, but say "replace the combiner with a meshinstance".

This is being tracked in https://github.com/godotengine/godot-proposals/issues/182.

[fire]

I updated to the latest manifold sources.

Still having trouble with original mappings from the triangles, but many things have been improved in the manifold code.

[elalish]

Agreed that's harder than it ought to be. I'm working on a new API that should simplify handling properties.

[fire]

@elalish Can you provide a way to associate an arbitrary set of data like? in the properties:

    struct Face {
        Vector3 vertices[3];
        Vector2 uvs[3];
        AABB aabb;
        bool smooth = false;
        bool invert = false;
        int material = 0;
    };

[elalish]

@fire can you be a touch more specific? We talk about how to do some of this using MeshRelation in #274. I really want to avoid prescribing what kinds of materials and properties can be associated with Manifold, so it can be used generically. I'm now thinking of expanding on the properties and triProperties vectors I have to automatically interpolate and output them as part of MeshGL. I'm hoping that will make it easier than the barycentric MeshRelation, without actually having to name any properties.

In your example, what would smooth represent? Do you really want non-indexed vertices? Why an AABB per triangle? It's pretty easy to calculate from the verts.

[fire]

The problem is Godot in my questionable design has two ways to specify the same data so if the data is not written in a manifold internal format, it becomes randomly non-manifold and the entire tree of csg nodes gets lost (empty manifold)

So I was trying to think of ways to stick arbitrary data per face, per vertex, etc in the manifold data structures.

[elalish]

I wouldn't say it's a questionable design; it's what graphics drivers require, but yes it causes major headaches for manifolds. I believe I now have a good solution for a small amount of extra data (mergeFromVert and mergeToVert vectors) that will allow the manifold to be easily reconstructed from graphics-friendly structures: https://github.com/elalish/manifold/pull/290#discussion_r1033248128

I'm planning to make it as simple as possible to write out a GLB this way (ideally with an extension for this manifoldness data), so that engines can just use their GLB importers instead of having to write a special one for Manifold. It's not done yet, but it would be great to get your feedback on that PR as it progresses.

[fire]

@elalish what are your thoughts on running the https://github.com/hjwdzh/Manifold preprocess over the input mesh

[elalish]

@fire thanks, that's an interesting paper and algorithm. All manifold fixing algorithms are fundamentally heuristic, so I'd recommend testing it on quite a few bad meshes, especially characters which are often intentionally modeled with open meshes. I'm sure it'll give poor results sometimes, but I'm not sure how often. It'll also dramatically increase your vertex count.

I think it's a great idea to give it a try, but as with all fixing algorithms, please test for manifoldness first and only apply it if it fails (or simpler fixes don't work). A huge part of the problem with current workflows is that they will automatically do things like merging vertices in an attempt to fix manifoldness, but if the input was already manifold, it often breaks it instead.

[fire]

@elalish Did you stabilize the algorithm for triangle / vertex attributes? How is that going?

[elalish]

@fire Quite well, finally. Now would be a good time for API feedback if you have any. I need to finish my open PR, probably clean up a few remaining details, then I'll work on an npm v2.0 release.

[fire]

I am having some trouble. What is a commit that has the new api and works? My meshes have all been deemed non manifold,

[elalish]

It's really hard to help with that kind of a description. Can you show an example? Did it used to work? What are you attempting to do?

[elalish]

You can see for yourself that our tests are passing, perhaps you can write a test to catch the problem you're having?

[elalish]

@fire Any more information you can give me on this? I see you're now restoring materials, which presumably means you're now using MeshGL? For anything with discontinuous vertex properties, you'll need the mergeFromVert and mergeToVert vectors to maintain the manifoldness. Next I'm planning to build a helper to auto-generate those for a MeshGL based on tolerance welding, but that'll always be a heuristic process. Any examples you can give of what's giving you difficulty would help in my design of that.

[fire]

I am currently trying to release Godot Engine 4.0 so things have been hectic. Sorry for the lack of communication.

[fire]

Some interesting new development. https://github.com/SarahWeiii/CoACD was able to turn arbitrary meshes into a manifold meshes.

Not sure how, but something to do with openvdb.

Useful for supporting arbitrary meshes as input

[elalish]

@fire Yeah, I have quite a bit of familiarity with algorithms to make meshes manifold - I managed Microsoft's relationship with Netfabb when we were integrating their watertighting algorithms into our software. If you want to do it generically (like they did) you need a lot of heuristics and there's no avoiding having some very surprising results occasionally.

This library isn't about heuristics, but reliability. That said, I would like to help with meshes that are at least attempting to be manifold to get over the hump. I'm considering a function like enum MeshGL::Merge(float precision) that would generate the mergeFromVertex and mergeToVertex vectors by welding verts on open edges that are separated by less than precision. It would return a status of either AlreadyManifold, NowManifold, or NotManifold. This would make imported models with properties much easier to deal with.

However, it would help my development greatly if you could supply some example models that you were expecting to work and reported as NotManifold, so I can test this flow.

[yetigit]

@fire did a thing here https://github.com/yetigit/force-manifold

that uses just the openvdb part without the convex hull decomposition (which is still quality stuff); basically this is raymarching looking meshes being generated, guaranteeing that the mesh is a manifold. however often times the resolution is high

LEFT is the result RIGHT is the input

[elalish]

Thanks for sharing! Yeah, voxels are a reliable way to get manifoldness, though of course they kill the efficiency of a mesh. I'm almost done with my Merge function - curious how frequently it'll help.

[fire]

I decided to try for fun today and after vendoring thrust, glm, nanobind, quickhull subrepos I hit a roadblock with thrust requiring cuda..

I think I will have a hard time convincing the other Godot Engine maintainers that adding all these libraries is worth the load.

[pca006132]

but thrust doesn't require cuda? you can try to supply the thrust directory into manifold by setting FETCHCONTENT_SOURCE_DIR_THRUST. It should work. We are building it without cuda in the CI.

[fire]

I posted an updated branch to https://github.com/V-Sekai/godot/tree/vsk-csg-4.3.

Note that the default csg shapes aren't manifold so they tend to disappear on use in a csg tree.

Edited:

Something about the indexed meshes requirement breaks the vertex attributes if I try to add more.

[elalish]

Something about the indexed meshes requirement breaks the vertex attributes if I try to add more.

Happy to help if you can give a little more detail. When you have discontinuous vertex properties, you'll need the mergeFromVert and MergeToVert vectors in MeshGL to tell it how to stitch them to make a manifold. Alternately, you can use MeshGL.Merge() to generate these automatically, fixing gaps up to precision wide.

[fire]

Merge() helped significantly.

https://github.com/V-Sekai/godot/blob/549b24ca9b5fcd5be5b9b3c1923cb21816ba0e7c/modules/csg/csg_shape.cpp#L190

Now I am stuck on associating ??? ReservedIDs with each csg shape's Vector\<Ref\<Materials>> because each shape has it's own material index and it's unique to the vector of materials.

The above is a blender monkey and a mesh.

Edited:

Use the branch due to bugfixes.

[fire]

This is regular Godot Engine CSG for comparison to Manifold CSG

[elalish]

Looks like great progress!

Now I am stuck on associating ??? ReservedIDs with each csg shape's Vector<Ref> because each shape has it's own material index and it's unique to the vector of materials.

Yes, that's exactly what they're for. I like to make a map of originalID to Material (or material index). If each Manifold you create for input has a single material, you can just use manifold.OriginalID() to get the new entry for the map. If it has multiple materials, then you'll want to organize the triangles into runs of consistent material (like draw calls) and call Manifold::ReserveIDs() to get however many you need and put them in the meshGL.runOriginalID vector.

Does that make sense?

[fire]

The task would be to convert TypedArray\<Face> into runs of single materials from its current mixed materials. Then something.

The face struct looks like:

struct CSGBrush {
    struct Face {
        Vector3 vertices[3];
        Vector2 uvs[3];
        AABB aabb; // We calculate this directly.
        bool smooth = false;
        bool invert = false;
        int material = 0;
    };
    // ...
};

Then the task gets fuzzy after that.

[elalish]

Yes, so just sort your vector of faces by material. Then find the indices of that vector where the material changes. These values become meshGL.runIndex. Then get a sequence of originalIDs with int first = Manifold::ReserveIDs(numMaterials);. Make a mapping between these originalIDs and your materials. Then fill meshGL.runOriginalID with the ID that maps to your material for each sorted run.

When you get a MeshGL result after operations, the triangles are already sorted into these runs of consistent ID that you can use your map to return to material. Does that make sense?

[fire]

So using Merge() is incompatible with making my own runs right?