Ray Offset Precision utility + unit test

[devshgraphicsprogramming]

Description

Read this: https://www.pbr-book.org/3ed-2018/Shapes/Managing_Rounding_Error and implement ray-offsetting functions.

There should be a unit test which has "random" triangle models (BLAS) instanced in a TLAS and uses KHR_ray_query to shoot rays from random points on triangle surfaces in random directions over a hemisphere and check with GL_KHR_debug_printf that no self-intersection occurs after offsetting the ray.

As a bonus point, the test should perform a bisection search to establish the "optimal" offset and compare the utility generated offset and warn us if its much more suboptimal.

For even more points, implement your own "software intersection" to compare notes with the HW rayQuery.

Description of the related problem

There are multiple possible sources of error leading to self intersection.

Ray Origin Reconstruction (Reprojection to Point on Triangle)

Due to:

1. barycentric coordinate precision (can be f32 or unorm16)
2. barycentric interpolation formula precision (epsilons due to magnitude of triangle positions)
3. transform into world coordinates precision (epsilons due to multiplication of model-space ray-origin with 4x3 world-matrix)

Ray-Triangle Intersection

Due to:

1. ray direction precision (ray direction does not need to be normalized, remember that!) - only affects if the offset needs to be added or subtracted
2. ray transformation precision (mul of origin with 4x3 inverse-world-matrix and direction with upper 3x3 of same matrix)
3. ray intersection formula precision:
   • cross product to establish triangle normal
   • plane intersection formula
4. precision of model space triangle vertices or precomputed BLAS intersection helper data

Solution proposal

Need to find out what primitive intersection algorithm Nvidia's RTX (and ideally Intel and AMD) is using to compute the errors in ray-triangle intersection.

If no info available we can assume that a ray-plane intersection is used to determine ray-hit distance (so it doesn't matter is Moller Trumbore or Woop/Pichler), Baldwin/Webber is not watertight so unlikely to be used in HW ray-triangle intersection units. However its important to know which vertex is used as "the anchor" (which is the origin of the triangle edge vectors), if no info available or this is different between IHVs we have to compute/pick the worst geometric normal bounds.

Then implement a function which returns:

• barycentrically interpolated world space position
• unnormalized geometrical normal (with magnitude of twice the area of the triangle)
• a single vec3 offset (to apply to position to prevent self intersection)

Given the following arguments/template parameters:

• barycentric coordinate precision [template/type/suffix]
• barycentric coordinate value
• anchor vertex index (for both barycentric reconstruction and intersection formula) [template/type/suffix - 0,1,2,unknown]
• vertex format precision (N-bit UNORM or 16/32bit SFLOAT) [template/type/suffix]
• vertex model-space positions
• world transform and inverse world transform as mat4x3

Later the sign of the dot product of the ray direction and raw-unnormalized geometrical normal can be used to determine whether the offset computed needs to be added or subtracted to the ray-origin.

Additional context

Revive this work: https://github.com/Devsh-Graphics-Programming/Nabla/tree/error_bounds_in_raytracing

[devshgraphicsprogramming]

How to structure the test

Ray cannot hit:

• its own spawning triangle
• any triangle that's a "convex" (none of the vertices are in the hemisphere of the originating triangle) "neighbour" (sharing at least one vertex)

This means that each triangle needs two compact lists of "convex neighbours" one for the front face and one for the backface.