search

ANTsX / ANTs

Advanced Normalization Tools (ANTs)
Apache License 2.0
1.18k stars 381 forks source link

Misalignment between Multimodal Templates #1104

Closed hermeckerogg closed 3 years ago

hermeckerogg commented 3 years ago

Hi all.

Describe the problem Please describe the problem.

Having some issues with the dimensions in a multivariate template I'm building. The modalities include the T1 from each subject, the Single Band Reference image from the fMRI timeseries, and the FA map from the DWI data computed with FSL.

I assumed it was an issue similar to the one I had a while back in this thread: https://github.com/ANTsX/ANTs/issues/451

However, even after padding in 50 voxels in all directions, it's still cutting off the bottom and posterior parts of the brain (see attached images, Template2 is the one with the problem).

I've seen something similar happen when the images get moved out of frame, when I double checked, there appear to be at least 2 of the 199 FA maps that meets this criteria (which I've since removed).

To Reproduce This is the SLURM script I ran to make it with the relevant inputs:

!/bin/bash

SBATCH --ntasks=1

SBATCH --nodes=1

SBATCH --cpus-per-task=3

SBATCH --partition=qTRDHM

SBATCH --job-name=MACIP_Template

SBATCH --mem-per-cpu=8192M

cd /data/mialab/users/tderamus/MACIP_Data/derivatives/TemplateWorkflow

antsMultivariateTemplateConstruction2.sh -d 3 -k 3 -y 0 -r 0 -z /data/mialab/users/tderamus/MACIP_Data/derivatives/TemplateWorkflow/Initializing_T1_in_MNI.nii.gz -z /data/mialab/users/tderamus/MACIP_Data/derivatives/TemplateWorkflow/Initializing_EPI_in_MNI.nii.gz -z /data/mialab/users/tderamus/MACIP_Data/derivatives/TemplateWorkflow/TemplateWorkflow/Initializing_FA_in_MNI.nii.gz -c 5 -o /data/mialab/users/tderamus/MACIP_Data/derivatives/TemplateWorkflow/MACIP_Template /data/mialab/users/tderamus/MACIP_Data/derivatives/TemplateWorkflow/MACIPpaths.csv

Outputs are attached, but the logs didn't appear to be saved.

System information (please complete the following information) CentOS Linux release 7.8.2003 (Core) NAME="CentOS Linux" VERSION="7 (Core)" ID="centos" ID_LIKE="rhel fedora" VERSION_ID="7" PRETTY_NAME="CentOS Linux 7 (Core)" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:centos:centos:7" HOME_URL="https://www.centos.org/" BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7" CENTOS_MANTISBT_PROJECT_VERSION="7" REDHAT_SUPPORT_PRODUCT="centos" REDHAT_SUPPORT_PRODUCT_VERSION="7"

CentOS Linux release 7.8.2003 (Core) CentOS Linux release 7.8.2003 (Core)

Cluster info via SLURM PARTITION AVAIL TIMELIMIT NODES STATE NODELIST qTRD up 5-08:00:00 2 mix , qTRD up 5-08:00:00 8 alloc ,,,,,,, qTRD up 5-08:00:00 11 idle ,,,,,,,,,, qTRDEV up 5-02:00:00 1 mix qTRDEV up 5-02:00:00 1 idle qTRDHM up 5-08:00:00 1 mix qTRDHM up 5-08:00:00 2 idle , qTRDGPUH up 5-08:00:00 2 mix , qTRDGPUH up 5-08:00:00 4 alloc ,,, qTRDGPUM up 5-08:00:00 2 mix , qTRDGPUM up 5-08:00:00 4 alloc ,,, qTRDGPUL up 5-08:00:00 2 mix , qTRDGPUL up 5-08:00:00 4 alloc ,,, qTRDGPU* up 5-08:00:00 19 alloc ,,,,,,,,,,,,,,,,,,

ANTs version information ANTs Version: 3.0.0.0.dev122-g65096 Compiled: Sep 27 2019 09:40:27

Additional information You think padding the original input images would help? The bounding boxes seem a little smaller in the FA maps. ANTsIssueFiles.zip

ntustison commented 3 years ago

Can you try taking out the FA maps and building a multivariate template from the two other modalities?

hermeckerogg commented 3 years ago

For troubleshooting purposes?

Part of what I was hoping to do is make an argument that using multimodal templates in a multimodal fusion data analysis would improve the results.

In a perfect world, the FA maps are registered to the FA template and each of the other modalities go to the other.

ntustison commented 3 years ago

For troubleshooting purposes?

Yes.

hermeckerogg commented 3 years ago

What came out with the FA maps reviewed and this command instead:

!/bin/bash

SBATCH --ntasks=1

SBATCH --nodes=1

SBATCH --cpus-per-task=3

SBATCH --partition=qTRDHM

SBATCH --job-name=MACIP_Template

SBATCH --mem-per-cpu=8192M

cd /data/mialab/users/tderamus/MACIP_Data/derivatives/TemplateWorkflow

antsMultivariateTemplateConstruction2.sh -d 3 -k 2 -y 0 -r 0 -z /data/mialab/users/tderamus/MACIP_Data/derivatives/TemplateWorkflow/Initializing_T1_in_MNI.nii.gz -z /data/mialab/users/tderamus/MACIP_Data/derivatives/TemplateWorkflow/Initializing_EPI_in_MNI.nii.gz -c 5 -o /data/mialab/users/tderamus/MACIP_Data/derivatives/TemplateWorkflow/MACIP_Template /data/mialab/users/tderamus/MACIP_Data/derivatives/TemplateWorkflow/MACIPpaths_NoFA.csv

ANTsIssuesFiles2.zip

ntustison commented 3 years ago

Okay, so based on this, I have a quick question: did you perform an intra-modality registration for each subject to align each subject's image data prior to running template construction?

hermeckerogg commented 3 years ago

I did not. I created an initial template registered to an MNI hoping it would be able to figure it out from there.

I did the same with a pediatric template recently and it turned out alright so I thought that wouldn't be a huge issue. PediatricTemplates.zip

I'll get that registration going and share the output as soon as I'm able.

ntustison commented 3 years ago

So, yeah, you need to pre-align all the images per individual prior to creating the template. And I made a mistake above---it should be "inter-modality" not "intra-modality." Sorry for the confusion. If it worked for your pediatric template, you just happened to get lucky.

hermeckerogg commented 3 years ago

Doing so now.

You think it would be safe to assume direction normalization to the time-series multi-modal EPI template is sufficient? Or you think it should be co-registered to the T1 and then moved anyway?

hermeckerogg commented 3 years ago

Good news and bad news.

Good news, it looks better for sure. Also I actually have the SLURM .out log this time.

Bad news, even when everything has been aligned (I moved the SBRef and FA maps to the T1 image), it is still shifting. AntsIssueFiles3.zip

ntustison commented 3 years ago

Please check your inputs and how you ran the algorithm. The size of the third template is different from the first two so obviously something is amiss in how you generated the results. The multi-modal images per subject should all be normalized to the same space and thus the same size. Note that the images between subjects can be of different sizes.

hermeckerogg commented 3 years ago

So it's throwing a new error this time.

Creating template /users/tderamus/MACIP_Data/derivatives/TemplateWorkflow/MACIP_Templatetemplate2.nii.gz from a population average image from the inputs.

Usage:

AverageImages ImageDimension Outputfname.nii.gz Normalize

Compulsory arguments:

ImageDimension: 2 or 3 (for 2 or 3 dimensional input).

Outputfname.nii.gz: the name of the resulting image.

Normalize: 0 (false) or 1 (true); if true, the 2nd image is divided by its mean. This will select the largest image to average into.

Example Usage:

AverageImages 3 average.nii.gz 1 *.nii.gz

Your initial template : /data/mialab/users/tderamus/MACIP_Data/derivatives/TemplateWorkflow/MACIP_Templatetemplate0.nii.gz[2 ] was not created. This indicates trouble! You may want to check correctness of your input parameters. exiting.

The inputs are still confusingly wonky so I'll upload them when I narrow it down, but have you seen anything like this before?

cookpa commented 3 years ago

What's the command to produce this error? Blindly guessing that this is a result of forgetting "-d 3"

hermeckerogg commented 3 years ago

This was the input to the template script, though I'm not sure that's the answer to your question:

antsMultivariateTemplateConstruction2.sh -d 3 -k 3 -y 0 -r 0 -z /users/tderamus/MACIP_Data/derivatives/TemplateWorkflow/Initializing_T1_in_MNI.nii.gz -z /users/tderamus/MACIP_Data/derivatives/TemplateWorkflow/Initializing_EPI_in_MNI.nii.gz -z /users/tderamus/MACIP_Data/derivatives/TemplateWorkflow/TemplateWorkflow/Initializing_FA_in_MNI.nii.gz -c 5 -o /users/tderamus/MACIP_Data/derivatives/TemplateWorkflow/MACIP_Template /users/tderamus/MACIP_Data/derivatives/TemplateWorkflow/Coreged_MACIPpaths.csv

cookpa commented 3 years ago

Thanks. So the dimension flag is there, but it's not finding your initial templates. From the same directory, can you verify all three exist? Eg

"ls /users/tderamus/MACIP_Data/derivatives/TemplateWorkflow/Initializing_T1_in_MNI.nii.gz"

The "population average of the inputs" should not be printed if an initial template is found. It also seems to have a problem finding the input data. I think @ntustison usually advises people to copy all the template input files to the same directory to avoid path troubles like this.

Macs store user data under "/Users" but the file system is normally case insensitive, so I'm not sure if that's the issue.

hermeckerogg commented 3 years ago

They're definitely there.

The dumps into a single directory was something I used in the past, but with increasingly large participant images I'm having to do trimming to save space.

If it happens again, I'll see if I can get some space to give that a shot.

ntustison commented 3 years ago

I'm assuming you fixed the image size and intra-subject multi-modal normalization issues I mentioned above. Can you update the link to the data you're using so that we can try to reproduce the error?

ntustison commented 3 years ago

Again, if you can limit the test case to something like three subjects (assuming it still fails), that would be ideal.

hermeckerogg commented 3 years ago

I did. Let me see if I can get some things compressed enough to upload.

hermeckerogg commented 3 years ago

Thought we had it this time but its still doing the same thing.

Code and everything I could fit attached.

Un/Fortunately, no SLURM errors or outputs this time, just the warp log, so it looks like it ran as intended.

ANTsIssuesFiles4.zip

ntustison commented 3 years ago

I'm guessing you have a redundant "TemplateWorkflow" in your call above, i.e., 

-z /users/tderamus/MACIP_Data/derivatives/TemplateWorkflow/TemplateWorkflow/Initializing_FA_in_MNI.nii.gz

Also, your EPI images are not well-aligned with the corresponding T1s, unlike your FAs. And since they're EPIs they tend to not be as useful for driving image registration. I would suspect their inclusion is what is contributing to poor results. Given this, it's not clear to me why the EPIs are even included in the template building process. Unless you can get better EPI registrations, I would simply build a T1/FA multivariate template and then warp the EPIs to that template space (via intra-subject registration to the T1s) if you needed an EPI representation in that space.

hermeckerogg commented 3 years ago

I'll address both and see what happens.

hermeckerogg commented 3 years ago

That redundant directory seems to have fixed it.

Apologies, as I wouldn't have even posted if I'd noticed it sooner.

Regarding removing the EPI, at some point I'm curious to see if there are major differences when normalizing an EPI image directly to an EPI template or doing it via the T1 to MNI via co-registration most pipelines do.

Is that something anyone in your group has experimented with?

MACIP_Templates.zip

ntustison commented 3 years ago

Is that something anyone in your group has experimented with?

Yes, in this work. Although the context is DWI, the motivation and rationale is the same.

hermeckerogg commented 3 years ago

So having delved into this more I have the following question.

Based on what I've understood from the paper and some of the other threads, to avoid circularity, I would need to apply the transforms from from the T1 to T1 template to all the modalities (Say Grey matter maps from T1 segmentations, EPI images, and FA images). If this is the case, I shouldn't normalized any of the FA maps directly to the multimodal FA template?

If this is the case, why make the FA template? Is it just contributing to the weighting?

ntustison commented 3 years ago

to avoid circularity, I would need to apply the transforms from from the T1 to T1 template to all the modalities

More accurately, you would calculate the transformation between the subject and the template using only the T1 images. You would apply the transforms to all the modalities to the template through this t1-template correspondence.

I shouldn't normalized any of the FA maps directly to the multimodal FA template

Right. Broadly speaking, circularity is selecting data based on the evaluation criteria. So if you're trying to align FA data for comparison across a population, you wouldn't want to compute the correspondence directly from those images as that correspondence is computed using quantities that are intrinsic to the evaluation measurements themselves. For example, mean squares is used as a surrogate for anatomical alignment. However, it directly reduces the voxelwise variance which is very much related to common statistical measures that are used (e.g., t-test).

If this is the case, why make the FA template? Is it just contributing to the weighting?

It's not necessary but it might help with alignment during template building and could potentially be used for presentation purposes.

hermeckerogg commented 3 years ago

Got it.

Thanks!

hermeckerogg commented 3 years ago

(Sorry to keep commenting on a closed issue).

But to make sure (I've been lurking on the Jacobian Determinant threads and it unfortunately isn't clear to me) I'm doing this right from a VBM perspective, I'd need to run:

  1. CreateJacobianDeterminantImage 3 Native_to_T1_Template1Warp.nii.gz Native_To_TemplateJacobian.nii.gz
  2. WarpImageMultiTransform GM_Map_in_Native_Space.nii.gz Unmodulated_GM_Map_Template_Space.nii.gz -R MACIP_TemplateT1.nii.gz Native_to_T1_Template0GenericAffine.mat Native_to_T1_Template1Warp.nii.gz
  3. Multiply (or is it divide) Unmodulated_GM_Map_Template_Space.nii.gz by Native_To_TemplateJacobian.nii.gz

I got the impression the affine wasn't necessary for the Jacobian based on the other threads.