Support snapping to secondary iModels

[pmconne]

Fixes https://github.com/iTwin/itwinjs-backlog/issues/1209.

imodel-native: https://github.com/iTwin/imodel-native/pull/877

Allow a TileTreeReference to provide a transform from the world coordinates of the viewport's iModel to those of some other iModel (the one identified by HitDetail.sourceIModel). This propagates through readPixels to HitDetail.transformToSourceIModel. AccuSnap forwards the transform if present when making the snap request to sourceIModel.

To test, enhance dta imodel attach to apply a transform to draw the attached iModel in the correct location. Previously it applied no transform, so the attached iModel would draw in an unexpected location but snapping would appear to work.

[pmconne]

@bbastings is the world-to-view matrix in SnapRequest going to do unexpected things? It's obtained from the viewport, whose definition of "world coords" comes from a different iModel than the one we're snapping to.

[bbastings]

@bbastings is the world-to-view matrix in SnapRequest going to do unexpected things? It's obtained from the viewport, whose definition of "world coords" comes from a different iModel than the one we're snapping to.

It's used to compare against the snap aperture. If world doesn't reflect the position of the element geometry in the view it's going to be an issue when the camera is on for certain. Checks like deciding if a snap is hot, or even choosing the "best" snap won't be correct.

Maybe you can use Matrix4d.createTransform from your display transform so you can use Matrix4d.multiplyMatrixMatrix? Matrix4d hurts my brain...might need help from Oslo.

EDIT: Oh yeah, also used to get boresite direction for picking faces/edges of BReps...anyway, it's important. :)

[pmconne]

Maybe you can use Matrix4d.createTransform from your display transform so you can use Matrix4d.multiplyMatrixMatrix? Matrix4d hurts my brain...might need help from Oslo.

Yeah, I did that. Testing with nearest snap with camera on, at some view orientations it will mostly track the surface interiors, though it tends to more eagerly jump to the edges than it would normally. At other view orientations it always jumps to the edges.

I don't see anything else in the snap request that would need a transform applied.

EDIT: with camera off, it seems to behave correctly.

[bbastings]

I think we may need to explain what we are trying to do Oslo and get some help. We essentially want to transform the view frustum so that it's effectively looking at the transformed model in it's original location...

[bbastings]

Here's my attempt at something:

let worldToView = hitVp.worldToViewMap.transform0; // Nitpick: this is a Matrix4d, don't call it worldToViewMap...
const toIModel = thisHit.transformToSourceIModel;

if (toIModel) {
  const frustum = hitVp.getWorldFrustum();
  frustum.transformBy(toIModel, frustum);
  const worldToNpcMap = frustum.toMap4d();
  if (worldToNpcMap) {
    const worldToViewMap = Map4d.createIdentity();
    worldToViewMap.setFrom(hitVp.viewingSpace.calcNpcToView().multiplyMapMap(worldToNpcMap));
    worldToView = worldToViewMap.transform0;
  }
}

Did this in notepad, buyer beware...

[pmconne]

Here's my attempt at something:

Seems to behave the same as what I pushed. Actual change:

     const toIModel = thisHit.transformToSourceIModel;
-    let worldToViewMap = hitVp.worldToViewMap.transform0;
+    let worldToViewMap = hitVp.worldToViewMap;
     if (toIModel) {
-      const toIModelMap = Matrix4d.createTransform(toIModel);
-      worldToViewMap = toIModelMap.multiplyMatrixMatrix(worldToViewMap);
+      const frustum = hitVp.getWorldFrustum();
+      frustum.transformBy(toIModel, frustum);
+      const worldToNpcMap = frustum.toMap4d();
+      if (worldToNpcMap) {
+        const worldToViewMap = Map4d.createIdentity();
+        worldToViewMap.setFrom(hitVp.viewingSpace.calcNpcToView().multiplyMapMap(worldToNpcMap));
+      }
     }

     const testPoint = toIModel?.multiplyPoint3d(thisHit.testPoint) ?? thisHit.testPoint;
@@ -710,7 +715,7 @@ export class AccuSnap implements Decorator {
       id: thisHit.sourceId,
       testPoint,
       closePoint,
-      worldToView: worldToViewMap.toJSON(),
+      worldToView: worldToViewMap.transform0.toJSON(),

[pmconne]

I tried extracting ViewState3d.computeWorldToNpc into a standalone function and transforming all the inputs. It no longer grabs onto the edges when snapping. Diff in case I missed something obvious.

     const toIModel = thisHit.transformToSourceIModel;
-    let worldToViewMap = hitVp.worldToViewMap.transform0;
+    let worldToViewMap = hitVp.worldToViewMap;
     if (toIModel) {
-      const toIModelMap = Matrix4d.createTransform(toIModel);
-      worldToViewMap = toIModelMap.multiplyMatrixMatrix(worldToViewMap);
+      let camera;
+      if (hitVp.view.is3d() && hitVp.view.isCameraOn) {
+        camera = new Camera({
+          ...hitVp.view.camera,
+          eye: toIModel.multiplyPoint3d(hitVp.view.camera.eye),
+        });
+
+        const worldToNpc = computeWorldToNpc({
+          inOrigin: toIModel.multiplyPoint3d(hitVp.viewingSpace.viewOrigin),
+          delta: toIModel.multiplyVector(hitVp.viewingSpace.viewDelta),
+          enforceFrontToBackRatio: !hitVp.view.displayStyle.getIsBackgroundMapVisible(),
+          camera,
+          viewRot: toIModel.matrix.multiplyMatrixMatrix(hitVp.viewingSpace.rotation),
+        });
+        if (worldToNpc.map) {
+          worldToViewMap = Map4d.createIdentity();
+          worldToViewMap.setFrom(hitVp.viewingSpace.calcNpcToView().multiplyMapMap(worldToNpc.map));
+        }
+      }

@dassaf4 would you or @saeeedtorabi be able to help us figure out how to transform the world-to-view map from the coordinate system of one iModel to another?

[pmconne]

@grigasp fyi flaked out on linux:

--[ FAILURE: presentation-full-stack-tests ]--------[ 1 minute 41.2 seconds ]--

  330 passing (2m)
  1 failing

  1) PresentationManager
       Cancel request
         cancels 'getNodes' request:
     AssertionError: expected promise to be rejected with 'PresentationError' but it was fulfilled with [ { key: { version: 2, …(5) }, …(3) } ]