Photonavigation and mesh Intersection in iTwinJS

[MarcBedard8]

Is your feature request related to a problem? Please describe. In PhotoNavigation, we need to identify photos that can "see" a particular point on a mesh. We would like to be able to detect if a vector traced from the camera position to the reference point intersect a mesh in order to discard photos where the reference piont is masked by a obstacle. (e.g.: a piont under a bridge is masked by the bridge and the photo should be discarded)

Describe the solution you'd like • Main missing function is pickFromRay. In Cesium, you specify a 3d ray (start + direction) and the function sends back the first intersection with the scene. For Photonavigation it is crucial so that we can apply visibility filtering and display correct photos to the user. • A way to compute a normal when clicking on a mesh. In cesium, there’s a function called pickPoint() where you gave the Point 2D click coordinate on the viewer (that is the pixel the user clicked) and it sends back the intersected 3d point if any. So in the PoC, when the user click at x;y, we also fake click at surrounding pixels so that we have more points on the mesh to compute quickly a normal. In iTwinJS, using doLocate is more difficult to copy this behavior since we don’t have access to the clicked pixel. That’s the two main things that seems to be missing in iTwinJS to emulate what we did in Cesium.

Describe alternatives you've considered

Additional context

[markschlosseratbentley]

@danieliborra will discuss

[danieliborra]

There is a "simple" way to do it using the GPU, by rendering the scene/depths from the point of view of the 3d_ray_start and reading the depth/normal from the render target. This is something I'd suggest prototyping to see if the performance is good enough.

[tm-zub]

Removed the milestone as per discussion with @markschlosseratbentley