DeepSSM Image-based Registration Problems

[iyerkrithika21]

There are few issues to the image-based registration (because we are assuming we don't have access to ground truth surfaces during inference) process followed for evaluation:

1. Function: get_image_registration_transform

   • Translation transform crashes here: itk_transform = SimpleITK.TranslationTransform()
   • The code should be changed to:

     # Get transform matrix
     parameter_map = result_transform_parameters.GetParameterMap(0)
     transform_params = np.array(parameter_map['TransformParameters'], dtype=float)
     if transform_type == 'rigid':
     itk_transform = SimpleITK.Euler3DTransform()
     elif transform_type == 'similarity':
     itk_transform = SimpleITK.Similarity3DTransform()
     elif transform_type!= 'tranlation':
     print("Error: " + transform_type + " transform unimplemented.")
     itk_transform.SetParameters(transform_params)
     itk_transform_matrix = np.eye(4)
     itk_transform_matrix[:3,:3] = np.array(itk_transform.GetMatrix()).reshape(3,3)
     if transform_type != 'translation':
     itk_transform_matrix[-1,:3] = np.array(itk_transform.GetTranslation())
     else:
     itk_transform_matrix[-1,:3] = transform_params.T

2. Should we expose other parameters for image registration to the users? This registration might require some parameter tuning for other complex anatomies.

3. 'rigid' is hardcoded here, so it always applies 'rigid' transformation even when others transform types are passed.

4. As per ITK documentation:

   • Rigid (itk::Euler3DTransform) applies a rotation and translation to the space given 3 euler angles and a 3D translation. Rotation is about a user specified center.
   • Similarity (itk::Similarity3DTransform) applies a rotation, translation and isotropic scaling to the space So do we need to call all three transforms as done in 'groom_val_test_images()' Lines: 396, 409, 422 ?

[akenmorris]

Great observations! Indeed when I try the translation transform, I get:

[2024-08-27 16:19:20.250] [error] Exception Caught: TypeError: Wrong number or type of arguments for overloaded function 'new_TranslationTransform'.
  Possible C/C++ prototypes are:
    itk::simple::TranslationTransform::TranslationTransform(unsigned int,std::vector< double,std::allocator< double > > const &)
    itk::simple::TranslationTransform::TranslationTransform(itk::simple::TranslationTransform const &)
    itk::simple::TranslationTransform::TranslationTransform(itk::simple::Transform const &)

At:
  /home/amorris/miniconda3/envs/shapeworks/lib/python3.9/site-packages/SimpleITK/SimpleITK.py(6597): __init__
  /home/amorris/sci/shapeworks/Python/DeepSSMUtilsPackage/DeepSSMUtils/image_utils.py(31): get_image_registration_transform
  /home/amorris/sci/shapeworks/Python/DeepSSMUtilsPackage/DeepSSMUtils/__init__.py(72): get_image_registration_transform
  /home/amorris/sci/shapeworks/Python/DeepSSMUtilsPackage/DeepSSMUtils/run_utils.py(400): groom_val_test_images

[akenmorris]

So do we need to call all three transforms as done in 'groom_val_test_images()' Lines: 396, 409, 422 ?

Don't we need to combine the 3 registrations? They are each done on different croppings of the reference image. Translation on full size, rigid on medium and then similarity on the smallest?

[akenmorris]

    if transform_type == 'rigid':
        itk_transform = SimpleITK.Euler3DTransform()
    elif transform_type == 'similarity':
        itk_transform = SimpleITK.Similarity3DTransform()
    else:
        print("Error: " + transform_type + " transform unimplemented.")

Won't your changes always error out on "translation"?

[iyerkrithika21]

Fixed the if statement.

Don't we need to combine the 3 registrations? They are each done on different croppings of the reference image. Translation on full size, rigid on medium and then similarity on the smallest?

However, the rigid and similarity transforms are applying translation and rotation in both of them. The only difference is similarity also applies scaling.
So, we are repeating rotation and translation in all three calls.

[akenmorris]

we are repeating rotation and translation in all three calls.

I agree, but they are successive refinements, right?

[iyerkrithika21]

I was unsure about the logic used to apply the different transforms with different padding values. If the goal is refinement, then okay.