confused about landmark handling

[w19787]

i have some questions about landmark handling. Your response will be greatly helpful to me. thanks in advance.

1. in data util of brain.json, the point1 and point2 input are always of -1.0. how the landmark can be calculated?

2. in the VTN paper,

   "For Landmark Distance, we perform the inverse of the linear part of the affine transform (which aligns the fixed image to the atlas) to the difference vector between warped landmark and landmark in the (aligned) fixed image, so that the length goes back to the coordinate defined by the original fixed image"

does the following implementation align with above statement:

it is quite hard time for me to understand (flow as im, pt1 as offset?)

self.trilinear_sampler([flow, pt1])

why cann't we apply flow to point2?

[zsyzzsoft]

1. There are no landmark annotations for brains in our datasets.
2. Flow maps image1 to image2. Applying flow to point2 needs an inverse operation, because you need to find which point in image1 corresponds to the given point2.

[w19787]

1. There are no landmark annotations for brains in our datasets.

is the VTN brain datasets different with Recussive-Cascade-Network? in VTN paper, there is landmark measurement for brain dataset validation.

2. Flow maps image1 to image2. Applying flow to point2 needs an inverse operation, because you

let me clarify my understanding. In our git project, img1 is fixed image, img2 is moving image. We find out a "flow" to reconstruct img2 to align with img1. the "flow" is the coordination mapping from warped_img2 to img2. so when we use following code, we actually find out the coordination of pt1 in origin img2. the landmark dist is calculated based on the distance in origin img2 space, instead of in warped img2 space. correct?

self.trilinear_sampler([flow, pt1])

thanks a lot.

[zsyzzsoft]

1. We later find that the landmarks used in the VTN paper are not accurate enough (given small displacements in brains) so we decide not to release them. I recommend following the settings of the ICCV paper.
2. Seems correct.

[w19787]

1. We later find that the landmarks used in the VTN paper are not accurate enough (given small displacements in brains) so we decide not to release them. I recommend following the settings of the ICCV paper.
2. Seems correct.

thanks so much for your time and kind response!!!

[w19787]

1. We later find that the landmarks used in the VTN paper are not accurate enough (given small displacements in brains) so we decide not to release them. I recommend following the settings of the ICCV paper.
2. Seems correct.

one more question about landmark. In this paper, the landmark unit of liver is in "voxel coordinates in image data array " (if i understand code and paper correct). but i read couple other papers, the landmark dist is in RAS coordinates which is using mm unit.

and other paper as follows:

it seems to me that landmark in mm unit is more comparable. why don't we use mm unit for landmark dist as well? thanks.

[zsyzzsoft]

It can be in mm unit if spacing information is available in those datasets. As we are registering image crops, directly measuring the pixel distance is a more convenient choice for us than back-projecting to the original space.

[w19787]

It can be in mm unit if spacing information is available in those datasets. As we are registering image crops, directly measuring the pixel distance is a more convenient choice for us than back-projecting to the original space.

got it!

[w19787]

@zsyzzsoft i try to use warped pt2 to calculate the landmark distance during pt1 and warped pt2.

but the result doesn't seems right. with the jacc and dice score increase, the landmark dist does't reduce accordingly.

would you kindly help to advice any issue on the modified landmark dist calculation?

[zsyzzsoft]

As we have discussed before, you need to find the warped_pt2 in img1 such that warped_pt2 + flow is nearest to pt2. Warping is not correct. Or you need to find an inverse field.