Precision problems on the GPU side

[Spiri0]

The Nanite version, which carries out the visibility check on the CPU side, works very precisely. The trident webgpu version with the visibility check in the gpu leads to meshlet overlaps. I don't see any difference between the functions on the CPU side in Nanite and those in the cull shader in Trident. But what is very different is the fact that the GPU is worlds more powerful, but with the 32 bit limitation it is also significantly more vulnerable when it comes to accuracy. This has presented me with challenges several times in my apps. @AIFanatic what do you think? Does a factor of 10e-10f really make sense in the GPU?

fn isMeshletVisible(meshlet: MeshletInfo, modelview: mat4x4<f32>) -> bool {
    var projectedBounds = vec4(meshlet.boundingSphere.xyz, max(meshlet.error.x, 10e-10f));
    projectedBounds = transformSphere(projectedBounds, modelview);

    var parentProjectedBounds = vec4(meshlet.parentBoundingSphere.xyz, max(meshlet.parentError.x, 10e-10f));
    parentProjectedBounds = transformSphere(parentProjectedBounds, modelview);

    let clusterError = projectErrorToScreen(projectedBounds);
    let parentError = projectErrorToScreen(parentProjectedBounds);
    return clusterError <= settings.dynamicLODErrorThreshold && parentError > settings.dynamicLODErrorThreshold;
}

This will probably not be the only place where precision will play a role.

[AIFanatic]

I’m not sure if that’s what’s causing overlaps. I need to check the errors of the meshlets to see if they are within the limits of 32 bits. A whole bunch of tests need to be done to ensure they work flawlessly. Although the priority would be on the meshlet generation itself. Currently the problem is the same nanite-webgpu has, which is the simplifier getting stuck. In the latest update I have used meshoptimizer simplifyWithAttributes and only lock the borders of the meshlets wrt to its group but I’m not sure if this is correct.