Points based registrations - one of two ways

[ctorti]

Hello,

I would appreciate some clarity in the implementation of points-/landmark-based registrations.

I echo the post here that the elastix manual (see 6.1.7 Corresponding points) is quite thin when it comes to describing points-based registration. Similarly, the SimpleElastix manual does not go into much detail.

I have found various posts that seem to cover two different approaches for points-based registration - which seem to fundamentally differ only in their 'Registration' parameter:

1. MultiMetricMultiResolutionRegistration (for example #70, #219, and #176)
2. MultiResolutionRegistrationWithFeatures (for example #10)

Following is my attempt at # 1:

RegImFilt = sitk.ElastixImageFilter()
RegImFilt.LogToFileOn()
RegImFilt.LogToConsoleOn()

ParamMap = sitk.GetDefaultParameterMap('affine')
ParamMap['Registration'] = ['MultiMetricMultiResolutionRegistration']
# Need to have a metric that proceeds CorrespondingPointsEuclideanDistanceMetric - see [1]:
ParamMap['Metric'] = ['AdvancedMattesMutualInformation', 'CorrespondingPointsEuclideanDistanceMetric']
# I've also tried setting a small non-zero weight for AMMI but keeping the weight for CPEDM dominant:
ParamMap['Metric0Weight'] = ['0.0']
ParamMap['Metric1Weight'] = ['1.0']
RegImFilt.SetParameterMap(ParamMap)

RegImFilt.SetFixedImage(FixIm)
RegImFilt.SetMovingImage(MovIm)
RegImFilt.SetFixedPointSetFileName("fiducials_fixed.txt")
RegImFilt.SetMovingPointSetFileName("fiducials_moving.txt")
RegImFilt.Execute()

RegIm = RegImFilt.GetResultImage()

[1] #70 and #176

where FixIm and MovIm are the fixed and moving 3D DICOM sitk images, and 'fiducials_fixed.txt' and 'fiducials_moving.txt' each have 3 indices for their respective images of the form:

index 3 x0 y0 z0 x1 y1 z1 x2 y2 z2

I chose the indices using the "Probe" tool in the OHIF-Viewer, and verified that they overlay as expected in a matplotlib plot.

The above code works but it takes >130 seconds to register a (512, 512, 192) image to a (256, 256, 176) image. As a comparison, a (mostly) default affine registration takes between 21-26 s with 'MaximumNumberOfIterations' = 256-512.

The OHIF-Viewer's origin is defined at the top left, and the indices are 1-indexed. I accounted for these both (i.e. 0-indexed and flipped the y indeces) so that they are in the ITK coordinate system. What's interesting is that when I use fiducials defined with origin at the top left I get what appears to be a decent (albeit slow) registration result. Yet when I account for ITK's origin at the bottom left, the resulting registered image had a left-right mirror perspective w.r.t. the fixed image! My understanding of @Borda 's post here is that he had the same findings.

Moving onto # 2, I replaced the line:

ParamMap['Registration'] = ['MultiMetricMultiResolutionRegistration']

with

ParamMap['Registration'] = ['MultiResolutionRegistrationWithFeatures']

but got the following error in elastix.log:

itk::ExceptionObject (000000136B3E74B0) Location: "unknown" File: C:\SimpleElastix\build_20200108\Elastix\Components\Registrations\MultiResolutionRegistrationWithFeatures\itkMultiInputMultiResolutionImageRegistrationMethodBase.hxx Line: 257 Description: itk::ERROR: MultiResolutionRegistrationWithFeatures(0000019040CC7180): ERROR: This registration method expects a MultiInputImageToImageMetric

I've not been able to find any info on 'MultiInputImageToImageMetric' either in the SimpleElastix or elastix manuals, or here. The closest that I've found from other posts is #70, but @tvessiere and @peterzzz12 reported a different metric called 'AdvancedImageToImageMetric'. In any case, no solution or explanation has thus far been provided unfortunately.

It would be very helpful to get an understanding of why attempt # 1 was so slow, and what am I missing in trying to implement attempt # 2. Is there some general guidance on how to approach points-based registration that someone can point me towards?

Thanks!