Warp and reorient a diffusion tensor image issue with "large" affine transformation, looking for help/confirmation

[valeryozenne]

Hi ANTs experts,

All data and scripts to reproduce the results is available at this link: https://filesender.renater.fr/?s=download&token=491f1769-c437-426b-b210-51f2d65a0bf2 Data were down sampled to lower resolution to reduce the size of the files.

Methods

I have been using ANTs for some months for reorienting tensors. I'm really happy to use it. The wikipage was really useful. https://github.com/ANTsX/ANTs/wiki/Warp-and-reorient-a-diffusion-tensor-image

I still have one surprising result when combining transformations (affine + warp) in some cases (when the affine transformation is significant). I wonder if you could have a look. I try to simplify as much the script, and to follow the convention and naming of the documentation : the script either apply on tensor images

• [COMPOSE MODE] : affine + warp transformations using the instruction for creating a compose transform in the wiki page
• [STEP BY STEP MODE] : affine transformation, then reorient then apply the warp transformation and then reorient, here interpolation are done twice.

I have N datasets, here I shared only 2, (see the results below)

• for dataset 1, [COMPOSE MODE] give wrongs results while [STEP BY STEP MODE] looks accurate,
• for dataset 2 results are right for both mode.

I'm suspecting either a mistake from my side or something related to the transformation itself, the affine transform make a significant shift. At the beginning, I also suspect the reference (fixed.nii.gz), or the physical space but the result is accurate if I apply transformations step by step.

Would it be possible the process, applyTransform + ReorientTensorImage uisng compose transform doesn't work for some specific tranformations ?

My full script (not include here) include Rigid + Affine + Warp , leading to 3 interpolation in step by step mode to get the correct orientation, I wish to do it in one step using compose transform .

Thanks in advance for your help.

Valéry

Results

I'm looking at the fiber orientation in the intraventricular septum, anatomy is simple, there are two distincts layers (in blue) fibers going up/down and (in green) fibers that are parallel to the tangent of the surface in the short axis view (transverse view).

For dataset 2 , comparison of COMPOSE MODE is similar to STEP BY STEP MODE, the difference is due to the fact that interpolation is done twice.

For dataset 1, COMPOSE MODE mode is wrong , while the STEP BY STEP MODE is accurate.

Figure were made with MRtrix as I don't know how to plot them in itksnap. Tensor images conversion from 4D to 5D format and 5D to 4D format has been carefully checked (in a previous post, thanks to Philip Cook) and then used in multiple datasets, acquisitions & samples and can be considered as valid.

[cookpa]

Thank you @valeryozenne. I simplified further using only the affine transform

NUM=1

TENSOR_5D_NII=Data/dt_5D_${NUM}.nii.gz
REFERENCE=Data/fixed_${NUM}.nii.gz

TRANFORM_SYN_GENERIC=Data/movingDT_ToFixed0GenericAffine_${NUM}.mat

AFFINE_AS_WARP=affineTest/affineAsWarp_${NUM}.nii.gz

mkdir affineTest

antsApplyTransforms -d 3 -o [ ${AFFINE_AS_WARP},1 ] -t ${TRANFORM_SYN_GENERIC} -r ${REFERENCE} -v

# These two are almost identical, suggesting the problem is in reorientation
antsApplyTransforms -d 3 -e 2 -i ${TENSOR_5D_NII} -o affineTest/dt_${NUM}_resampledAffine.nii.gz -t ${TRANFORM_SYN_GENERIC} -r ${REFERENCE} -v
antsApplyTransforms -d 3 -e 2 -i ${TENSOR_5D_NII} -o affineTest/dt_${NUM}_resampledAffineAsWarp.nii.gz -t ${AFFINE_AS_WARP} -r ${REFERENCE} -v

ReorientTensorImage 3 affineTest/dt_${NUM}_resampledAffine.nii.gz affineTest/dt_${NUM}_reorientedAffine.nii.gz ${TRANFORM_SYN_GENERIC}
ReorientTensorImage 3 affineTest/dt_${NUM}_resampledAffineAsWarp.nii.gz affineTest/dt_${NUM}_reorientedAffineAsWarp.nii.gz ${AFFINE_AS_WARP}

ImageMath 3 affineTest/dt_${NUM}_reorientedAffine_RGB.nii.gz TensorColor affineTest/dt_${NUM}_reorientedAffine.nii.gz 
ImageMath 3 affineTest/dt_${NUM}_reorientedAffineAsWarp_RGB.nii.gz TensorColor affineTest/dt_${NUM}_reorientedAffineAsWarp.nii.gz 

The results show clearly that the reorientation is different depending on whether the input is an Affine or Warp field.

@ntustison @jeffduda I think there might be a serious problem with ReorientTensorImage. I've uploaded a subset of @valeryozenne 's data to

https://upenn.box.com/s/p5z05nnmolzi8sw9i4lg5slrm5xdw1dv

for testing (enough to run my script above).

[cookpa]

I had tested for this issue in my orientation tests here

https://github.com/cookpa/antsDTOrientationTests

But it's possible that I didn't have the right combination of direction cosines / data transforms to see the problem.

[cookpa]

I re-ran the brain data tests from my repo above. I can't see any divergence between transforms stored as warp fields and those stored as matrices.

I've noticed some differences between my brain tests and Valery's data, but I don't know if any of them are significant:

1. My images were all RPI (per ITK-SNAP) / RADIOLOGICAL (per fslorient). The cardiac data is all LPI / NEUROLOGICAL. The orientation of tensors relative to the direction cosines can definitely cause orientations to be incorrect, but I don't understand why it would be incorrect for warps but OK for matrices.

2. The scale of the tensors is quite different; the mean diffusivity is much smaller in the cardiac data. This sometimes causes issues with numerical thresholds in code, but I am not aware of that being an issue in ANTs

3. The brain tests only used a rigid transform, not a full affine.

[valeryozenne]

Hi Philip ,

Thanks for your input. There is something I'm not sure to fully understand. I'm writing my understanding below to be sure.

• the issue is not due to the combination of affine + wrap, the issue is present with the affine transformation alone
• reorientation is different if the affine transformation is saved as .mat or as .warp.nii.gz.
• saving a affine transformation as a warp ("affineTest/affineAsWarp_${NUM}.nii.gz") means that we have a vector fields where all vectors are kind of "identical" (in case of translation or here there are at least no local deformation) , this is not the standard way I guess because it costs space on the disk. and a saving a affine transform with a quaternion is simpler to interpret.

Please don't hesite to indicate if I can do something ?

About the three points you mentionned, please find some comments (not sure it can help you) :

1)

• most of my DT datasets are really closed to the anatomical dataset except NUM=1. In case NUM=2 , I used the same code but reorientation looks goods, either maybe because the variations are too small. Comparing movingDTToFixed0GenericAffine${NUM}.mat from NUM=1 and NUM=2 might help ?

• My data have been acquired in Bruker, convert to Nifti , and convert to the "good itk orientation" by a let's try and see method (I admit) using convert command of mrtrix with the option -stride 1,2,3, option until the reorientation of my tensor were correct wathever the transformation I used. I did a lot of tests, using rigid , affine, and warp transform. Everything was working expect this issue when composing transformations. I already report the same problem in july 2020 (See the end of the topic: https://github.com/ANTsX/ANTs/issues/1036), but I wasn't fully sure of my work, so I give up. Now I applied my routines on different samples, and I'm more confident.

  • if I applied the .mat transformation , the reorientTensorImage , then apply the warp transformation and then reorientTensorImage everything looks fine,. All my routine include always such step by step to be sure now. In a way, it means that the itk orientation seems fine.

• . The fact that my data is stored in LPI is not wanted , I don't like this way of looking the data. Could you indicate how do you see it in PrintInfo ? What would be the correct message for RPI. How can I change it?

2) this is always true for cardiac diffusion

3) is it possible to run antsRegistration with r + s (without affine transform) ?

[cookpa]

the issue is not due to the combination of affine + wrap, the issue is present with the affine transformation alone

That looks to be the case for me. The results look quite different when calling ReorientTensorImage with the affine matrix vs calling it with the warp constructed from the affine.

reorientation is different if the affine transformation is saved as .mat or as .warp.nii.gz.

Right, essentially the same idea as above, but yes.

saving a affine transformation as a warp ("affineTest/affineAsWarp_${NUM}.nii.gz") means that we have a vector fields where all vectors are kind of "identical" (in case of translation or here there are at least no local deformation) , this is not the standard way I guess because it costs space on the disk. and a saving a affine transform with a quaternion is simpler to interpret.

It costs space and might be theoretically worse because it is less precise and cannot be inverted analytically. That's why we don't normally do it, and I would tolerate small differences in the results from doing that. But there should not be large differences.

The fact that my data is stored in LPI is not wanted , I don't like this way of looking the data. Could you indicate how do you see it in PrintInfo ? What would be the correct message for RPI. How can I change it?

It's not in ANTs, but in ITK-SNAP, which follows NIFTI convention.

http://www.itksnap.org/pmwiki/pmwiki.php?n=Documentation.DirectionMatrices

But I tried converting to RPI last night and it didn't seem to help.

if I applied the .mat transformation , the reorientTensorImage , then apply the warp transformation and then reorientTensorImage everything looks fine,. All my routine include always such step by step to be sure now. In a way, it means that the itk orientation seems fine.

This is puzzling, but helpful to know.

[cookpa]

Also yes, you can run rigid then deformable registration if you want. I often do this for nonlinear correction of intra-subject images, where there is not necessarily an affine component to the deformation.

[cookpa]

NUM=$1

outDir=affineTest${NUM}

TENSOR_5D_NII=Data/dt_5D_${NUM}.nii.gz
REFERENCE=Data/fixed_${NUM}.nii.gz

TRANFORM_SYN_GENERIC=Data/movingDT_ToFixed0GenericAffine_${NUM}.mat

AFFINE_AS_WARP=${outDir}/affineAsWarp_${NUM}.nii.gz

mkdir ${outDir}

antsApplyTransforms -d 3 -o [ ${AFFINE_AS_WARP},1 ] -t ${TRANFORM_SYN_GENERIC} -r ${REFERENCE} -v

# These two are almost identical, suggesting the problem is in reorientation
antsApplyTransforms -d 3 -e 2 -i ${TENSOR_5D_NII} -o ${outDir}/dt_${NUM}_resampledAffine.nii.gz -t ${TRANFORM_SYN_GENERIC} -r ${REFERENCE} -v
antsApplyTransforms -d 3 -e 2 -i ${TENSOR_5D_NII} -o ${outDir}/dt_${NUM}_resampledAffineAsWarp.nii.gz -t ${AFFINE_AS_WARP} -r ${REFERENCE} -v

ImageMath 3 ${outDir}/dt_${NUM}_resampledAffineNoReorient_RGB.nii.gz TensorColor ${outDir}/dt_${NUM}_resampledAffine.nii.gz
ImageMath 3 ${outDir}/dt_${NUM}_resampledAffineAsWarpNoReorient_RGB.nii.gz TensorColor ${outDir}/dt_${NUM}_resampledAffineAsWarp.nii.gz

ReorientTensorImage 3 ${outDir}/dt_${NUM}_resampledAffine.nii.gz ${outDir}/dt_${NUM}_resampledAffineReorientedAffine.nii.gz ${TRANFORM_SYN_GENERIC}
ReorientTensorImage 3 ${outDir}/dt_${NUM}_resampledAffineAsWarp.nii.gz ${outDir}/dt_${NUM}_resampledAffineAsWarpReorientedAffineAsWarp.nii.gz ${AFFINE_AS_WARP}

ReorientTensorImage 3 ${outDir}/dt_${NUM}_resampledAffine.nii.gz ${outDir}/dt_${NUM}_resampledAffineReorientAffineAsWarp.nii.gz ${AFFINE_AS_WARP}
ReorientTensorImage 3 ${outDir}/dt_${NUM}_resampledAffineAsWarp.nii.gz ${outDir}/dt_${NUM}_resampledAffineAsWarpReorientAffine.nii.gz ${TRANFORM_SYN_GENERIC}

for im in resampledAffineReorientedAffine resampledAffineAsWarpReorientedAffineAsWarp resampledAffineReorientAffineAsWarp resampledAffineAsWarpReorientAffine; do
  ImageMath 3 ${outDir}/dt_${NUM}_${im}_RGB.nii.gz TensorColor ${outDir}/dt_${NUM}_${im}.nii.gz 
done

Trying the same code for dt1 and dt2. The dt2 results don't look as different, but it might be because the rotation in the transform is smaller.

I also tried padding the dt1 images to rule out boundary problems. It doesn't change the results. My concern is that the method for extracting the local rotation from warp fields is not very good at capturing large rotations, and that is why the results for dt 1 look bad when the transforms are concatenated into a single warp.

[valeryozenne]

Yes, exactly this is not working for large rotation. I describe a bit the background. My sample are scanned in a cylindrical box. But the space is limited to put the whole heart (due to the size of the magnet bore). So in most of the samples, the long axis of the sample is the diagonal of the box resulting in a significant angle . (see my ugly figure below).

• Most of the time , DT and STI are scanned in the same session , so DT -> ST works. But sometime it is not possible due to the acquisition time. So I need to register DT->ST using affine + warp. Note that in all cases the ex-vivo samples are moving /shifting as the scan take 2-3 days. Therefore Registration is important and ANTs is important :-)

• But I also have the same problem when I applied the output of antsMultivariateTemplateConstruction2.sh to warp all the tensor in template space. If the rotation is too big , I need to process transform by transform (see (*) in the figure).

[ DT -> ST ]: Affine + Warp then [ ST -> Template ] :Rigid (to aligned them) + Affine + Warp Leading to 10 interpolations (5 for apply Transform and 5 for Reorient Transform) and potential approximations...

---

To go back to the code, I run antsRegistrationSyn.sh with -r and also -sr to check if it is related to the affine transform or not . The conclusion is identical (see results below). I can share the data. I can also shared the input of antsRegistrationSyn.sh if you wish to check the overall process.

• Apply rigid transform.mat and ReorientTensorImage [OK]

• Rigid.mat Versus Rigid.mat + Warp.nii.gz in "step by step" mode [OK] ( Apply rigid transform and then ReorientTensorImage and then Apply warp transform and then ReorientTensorImage) Minor difference are visible at the blue/green interface, but it is perfectly normal as the rigid transformation doesn't depict the "true" transformation to match DT on anatomical images.

• Rigid.mat Versus Rigid.mat + Warp.nii.gz in compose mode [error] ( using affine save as a warp in the compose transform) . Reorientation is wrong.