Open Shinmera opened 8 months ago
Hm, that's unfortunate; are you using the triangles to build a convex hull? I'd expect you only need the vertex positions, and a hull building algorithm would need to be robust to duplicate vertices and redundant vertices inside the hull (i.e., the hull {a,b,c,d} should come out equivalent to the convex hull of {a,b,c,d,a}, which should also come out equivalent to {a,b,c,d, (a+b+c+d)*0.25}.
It might be possible at export time, to build the convex hull and swap in the representation - Blender already has some functionality to compute convex hulls, though, I don't know if there's any caveats regarding tolerances or dropping features.
The hill-climbing algorithm needs vertex adjacency information in order to function, so the face index array is necessary to be correct for it to deliver accurate results.
The quickhull algorithm is a very well-known way to generate a minimal convex hull (along with the face index array) for a set of vertices.
So it seems Blender does some weird stuff when generating a convex hull, namely it tries to preserve UVs and normals, leading to a mesh with separated vertices, rather than a connected contiguous 2-manifold convex-hull. The exporter seems to preserve this, which causes issues when the engine wants to assume that the mesh pointed to in the glTF is actually a pure, single convex hull.
I came across this when implementing a hill-climbing optimisation and just assumed the input data was correct. Hill-climbing requires the mesh to be watertight, and when it isn't delivers wrong results. Took me quite a while to chase this down.
As a workaround I'm running the quickhull algorithm over the mesh in-engine now, but it would be nice to be able to safely avoid that computation.