Authors: John C. Williams, MS, Srineil Nizambad, BS, Yash Patel, MS, Philip Tubiolo, MS, Mario Serrano-Sosa, PhD, Karl Spuhler, PhD, Jared X. Van Snellenberg, PhD, and Chuan Huang, PhD.
Authors/developers of original CNN Striatal Segmentation Python code (https://github.com/MIRA-Lab-stack/Striatal_Segmentation): Mario Serrano-Sosa, PhD, Karl Spuhler, PhD, and Chuan Huang, PhD.
Table of Contents:
I. SUMMARY
II. RUNNING INSTRUCTIONS
III. PIPELINE STEPS
IV. UPDATE
V. REQUIRED DEPENDENCIES, PYTHON VERSION, AND OTHER FILES
This is a pipeline that produces CNN-based segmentations of the striatal regions of the brain for given structural and functional MRI images the user provides, using the algorithm described in:
Serrano-Sosa M, Van Snellenberg JX, Meng J, Luceno JR, Spuhler K, Weinstein JJ, Abi-Dargham A, Slifstein M, Huang C. Multitask Learning Based Three-Dimensional Striatal Segmentation of MRI: fMRI and PET Objective Assessments. J Magn Reson Imaging. 2021 Nov;54(5):1623-1635. doi: 10.1002/jmri.27682. Epub 2021 May 10. PMID: 33970510; PMCID: PMC9204799.
The original Python code (https://github.com/MIRA-Lab-stack/Striatal_Segmentation), which was slightly modified for usability on other systems (e.g., removing hard-coded paths), was developed by Mario Serrano-Sosa, PhD and the Medical Image Research and Analysis (MIRA) Labratory, directed by Chuan Huang PhD, Associate Professor of Radiology at Emory University (previously at Stony Brook University School of Medicine).
This toolkit allows a user to utilize this CNN striatal segmentation, handling user inputs and performing all required image manipulations. The user provides a native or MNI-space structual image, a brain mask in the same space, and, optionally, a BOLD fMRI template image to reslice the output into. Outputs are returned in the input anatomical space and both anatomical and BOLD resolutions (latter optional).
The toolkit was developed by John C. Williams, Srineil Nizambad, Yash Patel and Jared X. Van Snellenberg, at the Cognitive Neuroscience and Psychosis Lab at Stony Brook University School of Medicine.
The striatal regions of interest (ROIs) segmented are the: ventral striatum (VST), pre-commissural putamen (prePU), post-commissural putamen (postPU), pre-commissural caudate (preCA), post-commissural caudate (postCA).
The user provides a T1-weighted structural MRI image in ACPC orientation, a corresponding brain mask in ACPC orientation, and, optionally, a BOLD functional MRI template image, warp image and fnirtSourceT1 image for use in reslicing and warping the outputs to BOLD resolution in MNI space.
The following outputs are produced: a NIfTI image containing segmentations that can be overlaid on the T1 image and 10 separate NIfTI images for right and left hemispheric divisions of each of the 5 striatal ROIs segmented, in T1-based resolution. If the optional BOLD fMRI template image was specified, an additional NIfTI image containing segmentations that can be overlaid on the fMRI image and 10 separate NIfTI images for right and left hemispheric divisions of each of the 5 striatal ROIs segmented are produced, in BOLD-based resolution in MNI space.
The main operating script of the pipeline is main_CNNStriatalSegmentation.m. This script is called by CNNStriatalSegmentation_example_script.m, which has a set of parameters that the user is instructed to adjust therein. Full running operations are discussed in the second section, RUNNING INSTRUCTIONS.
SPM12 and wb_command must be on your MATLAB path for this to work. You can find the links to these dependencies in the running instructions.
INPUTS:
<T1 template filename>
(e.g., ...MNINonLinear/T1w_restore_brain.nii)<brainmask template filename>
(e.g., ...MNINonLinear/brainmask_fs.nii)<Segmentation Output Directory>
(e.g., /mnt/drive/outputdir)<Caudate Mask>
(e.g., ...caudateMask.nii)<Putamen Mask>
(e.g., ...putamenMask.nii)OPTIONAL INPUTS :
<BOLD template filename>
(image to reslice (resample) to, can be a BOLD image; e.g., ...MNINonLinear/Results/RSFC_fMRI_1/RSFC_fMRI_1.nii)<warpPathFileName>
(the warp from AC-PC aligned, distortion corrected, bias field corrected, native subject space to MNI space .../acpc_dc2standard.nii.gz)<fnirtSourceT1path>
(the source images for the FNIRT normalization from subject to MNI space - the non-skull-stripped T1 images, AC-PC aligned, distortion corrected, bias field corrected T1w image .../T1w_acpc_dc_restore.nii.gz)OUTPUTS*:
OPTIONAL OUTPUTS (if BOLD functional MRI, warpPath and fnirtSourceT1path are provided):
Several intermediates are also generated and discussed in PIPELINE STEPS.
EXAMPLE:
main_CNNStriatalSegmentation('T1_acpc_template_brain',T1_acpc_template_brain,...
'template_acpc_brainmask',template_acpc_brainmask,...
'segmentation_outputs_directory',segmentation_outputs_directory,...
'caudateMask', caudateMask,...
'putamenMask', putamenMask,...
'BOLD_template_image',BOLD_template_image,...
'warpPathFileName', warpPathFileName,...
'fnirtSourceT1path',fnirtSourceT1path);
Adjust the necessary parameters in CNNStriatalSegmentation_example_script.m; supply the paths for the files and subfolders as instructed therein and as described below.
For each run of the pipeline involving different subjects, the following are required and the paths must be adjusted: T1_acpc_template_brain, template_acpc_brainmask, segmentation_outputs_directory, caudateMask and putamenMask. The following input is optional: BOLD_template_image, warpPathFileName, fnirtSourceT1path.
acpc_dc2standard.nii.gz
in HCP MPP preprocessed data.T1w_acpc_dc_restore.nii.gz
in HCP MPP preprocessed data.The final anatomical resolution segmentation mask is named: anatRes_templateSpace_striatalCNNparcels.nii. The 10 separate hemispheric-specific ROI images produced are named: anat_left_prePU.nii,anat_right_prePU.nii,anat_left_preCA.nii,anat_right_preCA.nii,anat_left_postCA.nii,anat_right_postCA.nii,anat_left_postPU.nii,anat_right_postPU.nii, anat_left_VST.nii,anat_right_VST.nii.
If the optional BOLD fMRI template image, warpPath input and fnirtSourceT1 input is specified, the final BOLD fMRI resolution segmentation mask in MNI space is additionally produced and named: BOLDRes_templateSpace_striatalCNNparcels_WARPED.nii. The 10 separate hemispheric-specific ROI images produced are named: bold_left_prePU.nii,bold_right_prePU.nii,bold_left_preCA.nii,bold_right_preCA.nii,bold_left_postCA.nii,bold_right_postCA.nii,bold_left_postPU.nii,bold_right_postPU.nii, bold_left_VST.nii,bold_right_VST.nii.
a)The segmentations are resliced according to the resolution of the original T1 weighted MRI image input using 7th degree spline interpolation in SPM. The first final output, anatRes_templateSpace_striatalCNNparcels.nii, is generated.
b) Now the corrections in the anatRes_templateSpace_striatalCNNparcels.nii is made using the user inputted Caudate and Putamen masks. The correction is based on anatomical Boundaries and adjacency of voxels. This process involves:
c) Through the subfunction getseparatedROIs, the 5 whole-brain ROIs segmented in the anatRes_templateSpace_striatalCNNparcels.nii are separated and split between the left and right hemispheres to produce 10 hemispheric-specific ROIs, which are saved as separate NIfTI images: anat_left_prePU.nii,anat_right_prePU.nii,anat_left_preCA.nii,anat_right_preCA.nii, anat_left_postCA.nii,anat_right_postCA.nii,anat_left_postPU.nii,anat_right_postPU.nii, anat_left_VST.nii,anat_right_VST.nii.
This is achieved by having the product of step 10a set equal to zero for all values not equal to the integer representing the nth ROI in consideration. Since the product of 10a is an image where each voxel is either assigned to an integer representing each of the 5 ROIs (1-5) or not assigned to any ROI (0), each ROI can be separated as aforementioned. For each whole-brain ROI, the right and left hemispheric divisions of the ROI can be captured by setting the image to zero at indices that represent negative XYZ coordinates (as gathered by tippVol) and positive XYZ coordinates, respectively.
(Optional)
a) If the user specifies a BOLD fMRI template image, Warp image and the fnirtSourceT1 image then an additional output image is generated from anatRes_templateSpace_striatalCNNparcels.nii leveraging the wb_command's functionality to reslice to BOLD resolution and warp to MNI space. This output is BOLDRes_templateSpace_striatalCNNparcels_WARPED.nii.
b) Similar to step 10c, ten ROIs are separated from the BOLDRes_templateSpace_striatalCNNparcels_WARPED.nii image and split between the left and right hemispheres to produce the following: bold_left_prePU.nii,bold_right_prePU.nii,bold_left_preCA.nii,bold_right_preCA.nii, bold_left_postCA.nii,bold_right_postCA.nii,bold_left_postPU.nii,bold_right_postPU.nii, bold_left_VST.nii,bold_right_VST.nii
INPUTS:
<T1 template filename>
(e.g., ...MNINonLinear/T1w_restore_brain.nii)<brainmask template filename>
(e.g., ...MNINonLinear/brainmask_fs.nii)<Segmentation Output Directory>
(e.g., /mnt/drive/outputdir)<Caudate Mask>
(e.g., ...caudateMask.nii)<Putamen Mask>
(e.g., ...putamenMask.nii)OPTIONAL INPUTS:
<BOLD template filename>
(image to reslice (resample) to, can be a BOLD image; e.g., ...MNINonLinear/Results/RSFC_fMRI_1/RSFC_fMRI_1.nii)<warpPathFileName>
(the warp from AC-PC aligned, distortion corrected, bias field corrected, native subject space to MNI space; e.g., .../acpc_dc2standard.nii.gz)<fnirtSourceT1path>
(the template T1 image used by FNIRT during preprocessing to generate the warp; e.g., Here the acpc aligned, distortion corrected, bias field corrected T1w image .../T1w_acpc_dc_restore.nii.gz)OUTPUTS, INCLUDING INTERMEDIATES:
1. striatalCNNrotated_`<T1 template filename>`.nii
2. striatalCNNrotated_`<brainmask template filename>`.nii
3. striatalCNNres_striatalCNNrotated_`<T1 template filename>`.nii
4. striatalCNNres_striatalCNNrotated_`<brainmask template filename>`.nii
5. CNN_striatal_python_output_intermediate.mat
6. raw_StriatalCNNparcels.nii
7. striatalCNN_unrotated_raw_StriatalCNNparcels.nii
8. anatRes_templateSpace_striatalCNNparcels.nii
9. anat_left_prePU.nii
10. anat_right_prePU.nii
11. anat_left_preCA.nii
12. anat_right_preCA.nii
13. anat_left_postCA.nii
14. anat_right_postCA.nii
15. anat_left_postPU.nii
16. anat_right_postPU.nii
17. anat_left_VST.nii
18. anat_right_VST.nii
19. `<Segmentation Output Directory>`/Intermediates/*intermediate files for clustering and removing holes*
(20-30 are OPTIONAL, dependent on whether a BOLD fMRI input, warpPath input and fnirtSourceT1 input are provided)
20. BOLDRes_templateSpace_striatalCNNparcels_WARPED.nii
21. bold_left_prePU.nii
22. bold_right_prePU.nii
23. bold_left_preCA.nii
24. bold_right_preCA.nii
25. bold_left_postCA.nii
26. bold_right_postCA.nii
27. bold_left_postPU.nii
28. bold_right_postPU.nii
29. bold_left_VST.nii
30. bold_right_VST.nii
This pipeline was tested using the following versions of Python and libraries:
Python: 2.7.5
Tensorflow: 2.10.0
Numpy: 1.23.2
Nibabel: 4.0.2
Scipy: 1.9.
SPM12 must be installed and on the MATLAB path prior to running the pipeline.
wb_command from Connectome Workbench