Would SIFT3D work in multi modality registration - Heart MRI/XRay?

[pkuppens]

I am doing research on the registration of a 3D MRI volume and an X-Ray image of a heart.

I've looked into DRR's, segmentations, rigid body transforms, etc., but now I stumbled upon SIFT, and SIFT3D. To me, it seems that SIFT3D would short-circuit a lot of these problems, and result in much faster algorithms, and I would like my algorithm in near-real-time for new X-Ray images.

But before spending a lot of unnecessary effort, I want to validate if my naive-optimistic assumptions are correct:

• key points can be detected in both X-Ray images and MRI volumes
• the key points can be matched, even if the volume shapes and orientations are different
• the key points can be matched, even if they are from different modalities (e.g. blood visibility is different between MRI and XRay), however, key points in transition between muscle and air could match more easily

Concerns:

• the features/key points in MRI and X-ray are too different
• key points blood/muscle will be mismatched to key points muscle/air
• rotation inside the 3D matrix should be similar between the two images
• 2D images can't be used to be matched to 3D volumes

Obviously, I can not expect full research from the investigators/readers, but a gut feeling or experiences from similar experiments are more than welcome.

[pkuppens]

Unfortunately, I can already comment that this will probably not work as good as I hoped:

The feature I am looking for is more or less the contour/border between the dark shade and the light in the top right, and those are completely different from the key points that are found by SIFT in the OpenCV 2D version already.