Correspondence term potentially using wrong gradients for mesh domain

[medakk]

The gradient of the XYZ attribute is currently calculated as an identity matrix(and optionally scaled).

I don't think this is correct for the mesh domain. Moving one unit in the x axis may not actually imply that you'll be at a position with the coordinate 1 unit away because of the underlying geometry. Additionally, since we are moving along some oriented triangle, the y and z attributes will also be affected ie: dx/dy is not 0. I think this should be changed to the "gradient of position", similar to what we did with the normals.

Here, the color represents x coordinate(blue -> yellow) and the black lines represent the direction of gradient of the x coordinate. Notice that it isn't necessarily a line pointing in the x axis:

@sheryjoe @akenmorris thoughts?

[akenmorris]

Are you sure it's any different for image domains?

This is only for the correspondence force, right? Shouldn't these forces for XYZ live only in euclidean space and be irrespective of the surface?

[medakk]

For image domain, I think the current version maybe just slightly incorrect. The correspondence term is asking the question: which direction does this particle need to move in order to increase the 'x' attribute. For image domain, this is just differentiating x over R3(3D real number space) and evaluates to [1,0,0]. It is slightly incorrect in that it does not consider image spacing, as far as I can tell. We may have been compensating for this with different attribute scales, but it can't handle anisotropy

For mesh domain, the correspondence term is again trying to figure out which direction to move in order to increase the "x" attribute. But we have to differentiate over the mesh to find this direction, not euclidean space.

I am seeing this issue where a single particle in 1 sample is in bad correspondence. With the current gradient, I believe the particle is trying to tunnel through the mesh to go to the correct location. This ends up doing nearly nothing because of the geodesic walk projecting the particle's update to the triangle its on

Shouldn't these forces for XYZ live only in euclidean space and be irrespective of the surface?

I don't think so, I think any gradient must be taken wrt the domain. The domain type should dictate how this gradient is calculated

[akenmorris]

Ok, that makes sense. Except I still don’t see a difference between mesh and image, aren’t they just two representations of the same surface?

We’re not doing a walk on the images, though, just snapping in place. Right? Is that the difference?

[medakk]

We’re not doing a walk on the images, though, just snapping in place. Right? Is that the difference?

Hm, I'm not too sure now that I think about it. I feel like the identity is a valid gradient for (isotropic) image domains, but the problem with a particle unable to escape bad correspondence may still arise.

[akenmorris]

But don’t we get the correct direction by projecting the update vector to the tangent plane?

I think the issue of bad correspondence is potentially unrelated.

[medakk]

I think the difference is that in image domain, any XYZ within bounds is a valid position(to differentiate over)

In mesh domain, it does not make sense to query values at points where there are no triangles.

-- Karthik Karanth

On Sun, Feb 28, 2021, 19:00 Alan Morris notifications@github.com wrote:

Ok, that makes sense. Except I still don’t see a difference between mesh and image, aren’t they just two representations of the same surface?

We’re not doing a walk on the images, though, just snapping in place. Right? Is that the difference?

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/SCIInstitute/ShapeWorks/issues/1064#issuecomment-787583288, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACQMOKO5EVTZ42TG6AQED6DTBLYL7ANCNFSM4YLQOXMQ .

[medakk]

But don’t we get the correct direction by projecting the update vector to the tangent plane?

Needn't necessarily, what if the particle wants to travel in the inverse direction of the normal?

But yeah, it's quite possible the bad correspondence is unrelated.

-- Karthik Karanth

On Sun, Feb 28, 2021, 19:24 Alan Morris notifications@github.com wrote:

But don’t we get the correct direction by projecting the update vector to the tangent plane?

I think the issue of bad correspondence is potentially unrelated.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/SCIInstitute/ShapeWorks/issues/1064#issuecomment-787590658, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACQMOKN62HF5ENLUDOVYIMLTBL3GPANCNFSM4YLQOXMQ .

[akenmorris]

Where should it go if the update is directly opposite the normal?

[medakk]

Stay in place, as I presume it currently does.

If we used gradient of positions instead, the particle would never want to go in the inverse direction of the normal(ie we avoid this situation entirely). The gradient will always be on the tangent plane

[medakk]

I am seeing this issue where a single particle in 1 sample is in bad correspondence. With the current gradient, I believe the particle is trying to tunnel through the mesh to go to the correct location. This ends up doing nearly nothing because of the geodesic walk projecting the particle's update to the triangle its on

For completeness: I implemented gradient of positions and it did not solve my correspondence problem(which makes sense in retrospect, a particle can still get stuck in an undesirable location even with gradient of positions)

[medakk]

wrt image domain

but it can't handle anisotropy

I was wrong about this. Its gradient of a position in world space wrt world space(NOT index space), so identity matrix is correct for image domain, irrespective of anisotropy. Only mesh domain needs to be changed

[sheryjoe]

@medakk @akenmorris is this still an issue?

[medakk]

No changes were made regarding this. This branch uses gradient of position for mesh domain:

https://github.com/SCIInstitute/ShapeWorks/tree/trygradposition

I saw worse results in some dataset at the time. Warrants further investigation

[akenmorris]

Moving this to "Would be nice to have". We can keep the discussion open, but no one is actively working on it.