TransformixImageFilter yields only zeros

Hi!

I ran into a problem when trying to apply transformations (rigid + non-rigid) obtained via elastixImageFilter onto a binary image (a segmented object) using simpleElastix in python. So a little explanation first: I'm using a this code to first rigidly and then non-rigidly register 3D ConeBeam CT and 3D clinical CT data:

elastixImageFilter = sitk.ElastixImageFilter()
parameterMapVector = sitk.VectorOfParameterMap()
param_rigid = sitk.GetDefaultParameterMap("rigid")
param_nonrigid = sitk.GetDefaultParameterMap("bspline")
parameterMapVector.append(param_nonrigid)

elastixImageFilter.SetParameterMap(parameterMapVector)
elastixImageFilter.SetFixedImage(CBCT_volume)
elastixImageFilter.SetMovingImage(CT_volume)
elastixImageFilter.Execute()
reg_result = elastixImageFilter.GetResultImage()

CBCT_volume and CT_volume are 32bit signed int sitk.Images with the same spacial resolution and isotropic voxel size (read with the sitk.ImageSeriesReader()). The registration works fine. When I now apply the computed transformation to a binary image (a structure in the CT_volume e.g. the lung or the heart) the resulting volume contains only zeros. I also changed the FinalBSplineInterpolationOrder to 0 as suggested in the manual.

transform_param_map = elastixImageFilter.GetTransformParameterMap()
transform_param_map[0]['FinalBSplineInterpolationOrder'] = ['0']
transform_param_map[1]['FinalBSplineInterpolationOrder'] = ['0']
transformixImageFilter = sitk.TransformixImageFilter()
transformixImageFilter.SetTransformParameterMap(transform_param_map)
transformixImageFilter.SetMovingImage(binary_volume)
transformixImageFilter.Execute()
reg_struct_result = transformixImageFilter.GetResultImage()

I already tried to cast the binary image from 8bit unsigned int to 32bit signed int or 32bit float, but still no change. The same even happens when I try to apply the transformation on one of the CT volumes. I made additional tests using only rigid registration, unfortunately with the same result. I attached the transformation, as well as the log files of the combined registration.

TransformParameters.0.txt - rigid TransformParameters.1.txt - nonrigid elastix_rigid_nonrigid.txt - elastix log transformix_rigid_nonrigid.txt - transformix log

Am I doing something essentially wrong?

Another issue that I encountered: When trying to read and use non-rigid transformation files via sitk.ReadParameterFile from the disk python crashes. Loading and using rigid transformation files works fine.

param_rigid = sitk.ReadParameterFile(rigid_param_file)
param_nonrigid = sitk.ReadParameterFile(non_rigid_param_file)
transformixImageFilter = sitk.TransformixImageFilter()

transformixImageFilter.AddTransformParameterMap(param_rigid)
transformixImageFilter.AddTransformParameterMap(param_nonrigid)
transformixImageFilter.SetMovingImage(ct_ds_iso)
transformixImageFilter.Execute()

Is this a known issue?

I'd be really grateful, if you could help me solve those issues! Thank you in advance!

Best, Lukas

Hi @lukfischer

Result image filled with zeros

From your rigid transformParameterFile you can see that your transform parameters are (TransformParameters 0.041724 0.014853 0.005392 -38.760743 -177.765402 -790.974481), which is pretty large for an image of (Size 465 465 175).

Do you know if your binary image has the same origin than your moving image? Because I suspect that the binary data is moved completely outside the image frame. The difference in the image origin often comes from the conversion from a medical format (lets say DICOM) to any other format that does not explicitly keeps these metadata.

I can see two (easy?) solutions:

Convert your fixed and moving images (at least the moving image) to the same format as its binary mask, then use this one for registration. This way the origin of the two volumes should be the same, hence the transformation should give the expected result.
I expect that the binary image is directly created from your moving image (I may be wrong, but otherwise I don't really get why you would use these specific transform parameters). So why don't you create the binary mask from the registered version of your moving image?

Crash when reading transformParameterFile

Does the crash happens exactly? is it after transformixImageFilter.Execute()? The Python stack trace could help here, or the transformix log file, or any details about the error itself.

And just in case, it reminds me of this issue, could you try the suggested solution if your error is similar?

Hi @PertuyF! Thank you very much for your fast reply! And sorry, if this is the wrong place to ask such questions (as they are not 100% code issue related). Btw. is there a different place to ask more application related questions? I just found the elastix mailing list and wasn't sure if questions about SimpleElastix should be put there. I also want to thank you all for working on this great tool/project! Awesome work!

Result image filled with zeros

Do you know if your binary image has the same origin than your moving image? Because I suspect that the binary data is moved completely outside the image frame.

You were of course right! The origins of my volumes were different:

CT Volume (-324.365234375, -455.365234375, -1027.5) - read via sitk.ImageSeriesReader()
CBCT Volume (-231.99920654296875, -231.99920654296875, -86.99359893798828)  - read via sitk.ImageSeriesReader()
Binary Volume (0.0, 0.0, 0.0) - read via pyDicom and some self written code for RT DICOM sets

I expect that the binary image is directly created from your moving image

Actually the binary images are not created by me but read in from a Radiotherapy DICOM Structure Set. I assume that at some point of the pipeline (within my code or even earlier during the generation of the set) the origin was not correctly passed on.

Convert your fixed and moving images (at least the moving image) to the same format as its binary mask, then use this one for registration.

After I changed the Origin of the binary volume to the same as the CT volume it works! Thank you so much! The registration result Origin is now the same as the one of the CBCT volume.

Crash when reading transformParameterFile

Does the crash happens exactly? is it after transformixImageFilter.Execute()?

The crash happens exactly when executing transformixImageFilter.Execute(). But as I wrote only when using the non-rigid parameter file or the combination of both. When using solely the rigid parameter file it works.

The Python stack trace could help here, or the transformix log file, or any details about the error itself.

Unfortunately I can't provide a python stack trace. Python crashes with the dreaded pop-up dialog "Python hast stopped working". (I'm on a windows 7 machine). Adding a try/except block around transformixImageFilter.Execute() didn't help either. The transformix log indicates that there is a problem with ReadFromFile(), or at least the log ends there:

Installing all components.
InstallingComponents was successful.

ELASTIX version: 4.801
Command line options from ElastixBase:
-out      D:\SOME_PATH\elastix_logs/
-threads  unspecified, so all available threads are used
-def      unspecified, so no input points transformed
-jac      unspecified, so no det(dT/dx) computed
-jacmat   unspecified, so no dT/dx computed

Reading input image ...
  Reading input image took 0.000030 s
Calling all ReadFromFile()'s ...

And just in case, it reminds me of this issue, could you try the suggested solution if your error is similar?

I read through that issue, Thx! I do not use the InitialTransformParametersFileName function and also add the parameter files one by one via AddTransformParameterMap(). I also set the moving value SetMovingImage(ct_ds_iso).

Again thank you so much for your help!

Best, Lukas

Hi Lukas, it is a little difficult to tell from you answer, did you try to delete InitialTransformParametersFileName from TransformParameters.1.txt?

Hi @kaspermarstal !

Thank you, I completely overlooked that I should remove it from the actual transformation file. When I remove it, it actually works!

param_rigid = sitk.ReadParameterFile(rigid_param_file)
param_nonrigid = sitk.ReadParameterFile(non_rigid_param_file)
del param_nonrigid["InitialTransformParametersFileName"]
transformixImageFilter.AddTransformParameterMap(param_rigid)
transformixImageFilter.AddTransformParameterMap(param_nonrigid)
transformixImageFilter.SetMovingImage(ct_ds_iso)
transformixImageFilter.Execute()

EDIT: But, if I load the transformation files, the registration itself produces empty volumes again. And this time I checked the Origins. So basically, if I use transform_param_map = elastixImageFilter.GetTransformParameterMap() I can apply the transformation on my binary images, but if I load the parameters and add them to the transformixImageFilter like this

transform_param_map = sitk.VectorOfParameterMap()
transform_param_map.append(param_rigid)
transform_param_map.append(param_nonrigid)
transformixImageFilter.SetTransformParameterMap(transform_param_map)

it does not work.

Is there a way to tell, if both transformation parameter files were actually applied during transformation? I cant find a hint about that in the transformix log file. transformix_rigid_nonrigid_log.txt

Thanks again!

SuperElastix / SimpleElastix