Control low-spatial-frequency and high-amplitude deformations for nonlinear transformateions

[TheChymera]

I am having this issue (unsure if it is a bug, a missing feature, missing documentation, or inaccessibility of documentation) whereby the nonlinear transformation phase of an antsRegistration call ends up severely warping my image.

In most cases the transformation looks acceptable:

But in some it does not:

Interestingly this seems to only happen with specific moving images (I have been unable to determine the common denominator) and specific fixed images (e.g. the first but not the second one of our templates):

I assume this has to do with the black border, but I was unable to test, because at this point the templates are pretty much “given”.

In my quest to resolve this I have manually iterated through countless parameter variations based on the instructions here, here and on incremental empirical evidence. It has boiled down to this:

antsRegistration --float --collapse-output-transforms 1 --dimensionality 3 --initial-moving-transform [ /usr/share/mouse-brain-atlases/ambmc_200micron.nii, /mnt/overflow/test_ambmc/test2/generic_work/_ind_type_4/s_biascorrect/sub-4004_ses-ofM_acq-TurboRARElowcov_T2w_corrected.nii.gz, 0 ] --initialize-transforms-per-stage 0 --interpolation Linear --output output_ --transform Rigid[ 0.1 ] --metric MeanSquares[ /usr/share/mouse-brain-atlases/ambmc_200micron.nii, /mnt/overflow/test_ambmc/test2/generic_work/_ind_type_4/s_biascorrect/sub-4004_ses-ofM_acq-TurboRARElowcov_T2w_corrected.nii.gz, 1, 64, Regular, 0.3 ] --convergence [ 1000x1000x1000, 1e-14, 30 ] --smoothing-sigmas 2.0x1.0x0.0vox --shrink-factors 4x2x1 --use-estimate-learning-rate-once 0 --use-histogram-matching 1 --masks [ /usr/share/mouse-brain-atlases/ambmc_200micron_mask.nii, NULL ] --transform Affine[ 0.1 ] --metric MI[ /usr/share/mouse-brain-atlases/ambmc_200micron.nii, /mnt/overflow/test_ambmc/test2/generic_work/_ind_type_4/s_biascorrect/sub-4004_ses-ofM_acq-TurboRARElowcov_T2w_corrected.nii.gz, 1, 64, Regular, 0.3 ] --convergence [ 200x100, 1e-08, 10 ] --smoothing-sigmas 1.0x0.0vox --shrink-factors 1x1 --use-estimate-learning-rate-once 0 --use-histogram-matching 1 --masks [ /usr/share/mouse-brain-atlases/ambmc_200micron_mask.nii, NULL ] --transform SyN[ 1e-08, 35.0, 0.0 ] --metric MI[ /usr/share/mouse-brain-atlases/ambmc_200micron.nii, /mnt/overflow/test_ambmc/test2/generic_work/_ind_type_4/s_biascorrect/sub-4004_ses-ofM_acq-TurboRARElowcov_T2w_corrected.nii.gz, 1, 16, Regular, 0.8
] --convergence [ 300, 1e-64, 10 ] --smoothing-sigmas 0.0vox --shrink-factors 1 --use-estimate-learning-rate-once 0 --use-histogram-matching 0 --masks [ /usr/share/mouse-brain-atlases/ambmc_200micron_mask.nii, NULL ] --winsorize-image-intensities [ 0.05, 0.95 ]
 --write-composite-transform 1

I use MI here to have the operation performed more rapidly. Adding CC+MI improves the results but does not sort out the issue completely. At any rate, this kind-of works, but not really. Please note that the issue is with the SyN phase alone, everything else works great.

More generally I have observed that:

• histogram_matching=False is better
• MI+CC is better than CC alone, which is better than MI alone

I can't really observe a reliable impact of editing the syn parameters, however. Past work indicates that since I work with mouse brains (having a much smaller scale than human brains, about 1:10 length units), I should reduce the first parameter by a factor of 10, but even reducing it by a factor of 10^8 seems to have done little to address the issue.

[gdevenyi]

Suggestions

• Fix your bias field with N4 first
• Do a fully dense Rigid/Similarity/Affine before SyN with more blur levels
• Really, use CC.
• Don't winsorize your histogram

I have never had any issues with the default SyN parameters with Rats or Mice.

[stnava]

also - if you want to increase regularization, set the totalFieldVarianceInVoxelSpace > 0

the 1e-8 step sizes you are using are ... unusual. more like 0.1 - this is in voxel space. you may need to do brain extraction in new subjects. but maybe i am misinterpreting the pictures.

sharing problematic data always helps.

On Wed, Mar 13, 2019 at 9:40 AM Gabriel A. Devenyi notifications@github.com wrote:

Suggestions

• Fix your bias field with N4 first
• Do a fully dense Rigid/Similarity/Affine before SyN with more blur levels
• Really, use CC.
• Don't winsorize your histogram

I have never had any issues with the default SyN parameters with Rats or Mice.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ANTsX/ANTs/issues/734#issuecomment-472425823, or mute the thread https://github.com/notifications/unsubscribe-auth/AATyfuPhJowq0qetyL-5gJEPgZaYKrCwks5vWP-9gaJpZM4bs5dE .

[TheChymera]

thank you for your speedy replies :)

@gdevenyi

• the bias field is fixed, just that I apply the warp to the original image.
• But I do do Rigid/Affine before, as for blur levels I came to notice that more of them meant more of the low-spatial-frequency deformations.
• I know, but when I have to do manual machine learning it really helps to do it faster.
• I have tried this out, but it sometimes ends up failing with:

WARNING: In /var/tmp/portage/sci-biology/ants-2.3.1/work/ants-2.3.1_build/ITKv5-install/include/ITK-5.0/itkObjectToObjectMetric.hxx, line 529
MeanSquaresImageToImageMetricv4 (0x56c04d0): No valid points were found during metric evaluation. For image metrics, verify that the images overlap appropriately. For instance, you can align the image centers by translation. For point-set metrics, verify that the fixed points, once transformed into the virtual domain space, actually lie within the virtual domain. 

@stnava sorry for omitting that, please find a self-contained reproducible example here, there is an iteration step since the bad file doesn't always fail, but the good file always doesn't. Thanks for clarifying the unit for the SyN step sizes, that had been unclear to me. Sadly, increasing the totalFieldVarianceInVoxelSpace parameter does not seem to help as much as increasing updateFieldVarianceInVoxelSpace, and increasing both seems to nullify the slight positive effect acchieved by updateFieldVarianceInVoxelSpace. It is however entirely possible that my manual gradient descent has lead me into some local minimum which is entirely up the wrong tree. I would very much appreciate any suggestions from your side.

[gdevenyi]

I think you meant to have a different link or an attachment that's missing. I don't see a runnable example with data.

[TheChymera]

@gdevenyi oops, yes, that was an empty link, please check again now.

[gdevenyi]

Okay, I see what's going on.

antsRegistration has some quirks with non-isotropic voxels. Whenever I'm solving a registration problem, I always upsample my fixed image to the highest res voxel in isotropic spacing.

If you do this:

ResampleImage 3 badatlas.nii resample.nii.gz 0.075x0.075x0.075 0 4

And use resample.nii.gz as your fixed image, all the regular antsRegistration suggestions work as expected.

[TheChymera]

@gdevenyi interesting... but if you look at the headers, the images have the exact same level of anisotropy, so it really can't be just that, right?

Also, while completely removing the winsorization breaks the process, reducing the extent to which it is done has significantly improved the results, though I would again be very interested in further suggestions.

[gdevenyi]

Another comment regarding choice of atlas, your atlas is "extracted" while your moving image isn't. This in general causes pain because you have edges in one image while there's no edge in the other.

[gdevenyi]

Masking could fix this, but your mask is too generous so the edge is included in the registration.

[TheChymera]

@gdevenyi yes, I have noticed that the main difference between the atlas that always works and the one which occasionally doesn't it the black rim. But I had thought the moving image would just be contracted into the non-black space (and this also happens e.g. with the “good” moving image I sent you). Any idea what causes this intance and how it might be combated without addding a resampling node to the preprocessing or recreating the template?

[TheChymera]

@gdevenyi ok, so going back to your suggestion, I assume instead of

ResampleImage 3 badatlas.nii resample.nii.gz 0.075x0.075x0.075 0 4

you meant

ResampleImage 3 badmap.nii.gz _badmap.nii.gz 0.075x0.075x0.075 0 4

since the atlas is already isotropic. So I tried that and if anything it worsened the results, so I don't think it's the source of the problem or a good workaround.

[gdevenyi]

Hrm, I didn't realize the atlas was isotropic.

So, I rephrase the problem. The atlas resolution is lower than some/all of the moving image dimensions. So the first thing ants does is throw away useful data since the fixed image determines the space the work is done. Upsampling the atlas fixes this.

[stnava]

Yes. This is correct.

On Wed, Mar 13, 2019 at 7:52 PM Gabriel A. Devenyi notifications@github.com wrote:

Hrm, I didn't realize the atlas was isotropic.

So, I rephrase the problem. The atlas resolution is lower than some/all of the moving image dimensions. So the first thing ants does is throw away useful data since the fixed image determines the space the work is done. Upsampling the atlas fixes this.

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/ANTsX/ANTs/issues/734#issuecomment-472650793, or mute the thread https://github.com/notifications/unsubscribe-auth/AATyfvf07BnOyA1Eu6tD401Dx0NRk2Sxks5vWY8_gaJpZM4bs5dE .

--

brian

[ntustison]

@gdevenyi -- I would just rephrase a couple things:

• "antsRegistration has some quirks with non-isotropic voxels" --- I would suggest that the "quirks" associated with anisotropic sampling are general sampling issues as opposed to something particular to ANTs.

• Same thing with "So the first thing ants does is throw away useful data". These are general implementation issues which are not specific to ANTs.

We would prefer that people reading this forum not come away thinking that ANTs is somehow uniquely deficient with respect to these issues.

[gdevenyi]

Absolutely agree @ntustison , much better phrasing.

[stnava]

yes - it is more of an understanding issue on the part of the user. ants gives users a lot of freedom - and therefore increases the burden of understanding the underlying concepts.

brian

On Wed, Mar 13, 2019 at 8:07 PM Gabriel A. Devenyi notifications@github.com wrote:

Absolutely agree @ntustison https://github.com/ntustison , much better phasing.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ANTsX/ANTs/issues/734#issuecomment-472653929, or mute the thread https://github.com/notifications/unsubscribe-auth/AATyfoqJ0-kH2XQ093sl5glI5WMR92FBks5vWZLWgaJpZM4bs5dE .

[TheChymera]

The atlas resolution is lower than some/all of the moving image dimensions.

some, yes, the template is 0.2x0.2x0.2 whereas the structural scan is 0.167x0.075x0.65, the issue however, is that I am using the structural scan as an intermediary to register functional data (which is 0.312x0.281x0.65). Ideally I would not want to oversample the structural data too much due to both multiple comparison and memory concerns.

Still, I don't see how this would be the specific issue here, since over 50 other scans at the same resolution reliably register correctly, and it is only 2-3 where SyN introduces this strange roll/scale effect (which looks much more like some affine transformation).

Barring a change in the resolution of the target space, what else could I try? I basically just want to tell it to not make this kind of low spatial frequency high amplitude roll/scale. Mouse brains are much simpler than human brains, and I can reliably say that if the algorithm is moving 30% of the image more than 20 voxels away, its definitely wrong.

[stnava]

you have all the answers you need from prior discussion. these are non-trivial insights, so perhaps think through what they mean.

you are the only one who knows your data, its problems, etc. we are more or less giving general insights based on knowledge

gained from data that is different from yours.

the resolution of a statistical study is not dependent on the resolution of the transformation.

you can resample fmri derived data to whatever template resolution you prefer for final analysis.

On Wed, Mar 13, 2019 at 8:25 PM Horea Christian notifications@github.com wrote:

The atlas resolution is lower than some/all of the moving image dimensions.

some, yes, the template is 0.2x0.2x0.2 whereas the structural scan is 0.167x0.075x0.65, the issue however, is that I am using the structural scan as an intermediary to register functional data (which is 0.312x0.281x0.65). Ideally I would not want to oversample the structural data too much due to both multiple comparison and memory concerns.

Still, I don't see how this would be the specific issue here, since over 50 other scans at the same resolution reliably register correctly, and it is only 2-3 where SyN introduces this strange roll/scale effect (which looks much more like some affine transformation).

Barring a change in the resolution of the target space, what else could I try? I basically just want to tell it to not make this kind of low spatial frequency high amplitude roll/scale. Mouse brains are much simpler than human brains, and I can reliably say that if the algorithm is moving half the image more than 20 voxels away, its definitely wrong.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ANTsX/ANTs/issues/734#issuecomment-472657291, or mute the thread https://github.com/notifications/unsubscribe-auth/AATyfqr3cRgmBBKvmf06R06dpcZw89rRks5vWZcUgaJpZM4bs5dE .

[TheChymera]

sorry, I didn't intend to overwhelm you with specific details about the data, I just wanted to clarify the details surrounding the topic of resolution which came up.

At any rate, the process at hand is very simple, I want to make slight (which I can also define in precise voxel/length units) transformations to an ellipsoid to make it fit into another. I assume this functionality is available in ANTs. The answers I have gotten were:

• [x] Reduce the extent to which the image is winsorized, which helped (thanks @gdevenyi) but did not fully fix the issue.
• [x] Enlarge totalFieldVarianceInVoxelSpace, which will not work with the example I have compiled for you.
• [ ] Change the template/mask, which is infeasible.

Is there no ANTs parameter which reliably controls the amplitude of nonlinear transformations?

[stnava]

totalFieldVarianceInVoxelSpace will do it. your statement that it "will not work" is wrong.

brian

On Thu, Mar 14, 2019 at 9:07 AM Horea Christian notifications@github.com wrote:

sorry, I didn't intend to overwhelm you with specific details about the data, I just wanted to clarify the details surrounding the topic of resolution which came up.

At any rate, the process at hand is very simple, I want to make slight (which I can also define in precise voxel/length units) transformations to an ellipsoid to make it fit into another. I assume this functionality is available in ANTs. The answers I have gotten were:

• Reduce the extent to which the image is winsorized, which helped (thanks @gdevenyi https://github.com/gdevenyi) but did not fully fix the issue.
• Enlarge totalFieldVarianceInVoxelSpace, which will not work with the example I have compiled for you http://chymera.eu/debug/ANTs/regularization.tar.xz.
• Change the template/mask, which is infeasible.

Is there no ANTs parameter which reliably controls the amplitude of nonlinear transformations?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ANTsX/ANTs/issues/734#issuecomment-472845217, or mute the thread https://github.com/notifications/unsubscribe-auth/AATyfkHT2WT5-RrO9Z5htO6xKv7ntnyzks5vWkmggaJpZM4bs5dE .

[ntustison]

Yes, as Brian mentioned, the total field variance constrains the deformation. It might not be the magic parameter which fixes your single example but that is the ANTs parameter in question. Another alternative to try is the BSplineSyN transform which, I find, tends to produce a more elastic- type solution.

[gdevenyi]

While you're focusing on the SyN part of the registration it is my contention that this is a sign of a failure of the affine stage before it.

Here's my generic registration script for a affine+SyN registration, with its solutions to your good and bad images, using the upsampling. Without upsampling, the affine fails badly for this script, pointing to that being the source of your problems.

I suggest you QC your affine registrations as I do, by saving out an affine transformed image so you can see if your affine worked.

https://transfer.sh/ZcYku/regularization2.tar.xz

[gdevenyi]

[stnava]

yes - it is generally the case that when you see "odd deformations", it is due to a failed rigid or affine mapping.

brian

On Thu, Mar 14, 2019 at 10:19 AM Gabriel A. Devenyi < notifications@github.com> wrote:

[image: image] https://user-images.githubusercontent.com/3001850/54364121-a3b75e00-4642-11e9-9681-6750bcc2f9d3.png

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ANTsX/ANTs/issues/734#issuecomment-472878877, or mute the thread https://github.com/notifications/unsubscribe-auth/AATyfq9qTazpq7veAyABoVrwiq76tjuuks5vWlpugaJpZM4bs5dE .

[TheChymera]

@gdevenyi thanks for giving this a shot!

Could you explain what I should be noticing in the image which you posted? It seems to me like all cases work out without showcasing my initial issue. I have also downloaded and looked at your archive, and I cannot find anything to suggest the following:

Without upsampling, the affine fails badly for this script

Looking at output/badmap-affine.nii.gz and output/goodmap-affine.nii.gz, both look equaly well fitted to the template:

On a tangential note: I have also mentioned above that the error looks more affine-like than SyN-like to me, but I had done QC, and figured out that up until the SyN phase good/bad images tended to be equally well registered.

[gdevenyi]

I assumed that goodmap and badmap were two different input files.

I simply demonstrate that my script, using resample.nii.gz as a target both successfully register.

Here's the result if I use badatlas.nii.gz as the target.

[cookpa]

The domain of the fixed image defines the domain of the virtual space the computation is performed. So changing the spacing of the fixed image changes the interpretation of several antsRegistration command line parameters including shrink factors, smoothing kernels, regularization parameters, and metric sampling.

It makes sense that resampling the atlas would change results, though it might be possible to arrive at the same results by adjusting other parameters.

I managed to get reasonable results with

antsRegistration --float 1 --dimensionality 3 \
 --initial-moving-transform [ badatlas.nii, badmap.nii.gz , 0 ] --transform Affine[ 0.1 ] \
 --metric MI[ badatlas.nii, badmap.nii.gz, 1, 32, Regular, 0.25] --convergence [ 100x50, 1E-8, 10 ] \
 --shrink-factors 2x1 --smoothing-sigmas 1x0vox --transform SyN[ 0.1, 3.0, 1 ] \
 --metric CC[ badatlas.nii, badmap.nii.gz, 1, 4 ] --convergence [ 25, 1E-8, 10 ] \
 --shrink-factors 1 --smoothing-sigmas 0vox \
 --verbose \
 --output [ badMapFullReg, badMapDeformed.nii.gz ] 

As has been suggested above I did not use histogram matching or winsorization. I would be especially careful doing anything to the intensities if using mean squares metric, as the example script does. The other big variable, particularly if using different atlases, is going to be the initialization. If the geometric centers of the images align poorly, you might get a bad initial alignment.