nipype.interfaces.cat12.preprocess.CAT12Segment does not accept compressed `.nii.gz` files

JohannesWiesner commented 1 month ago

Summary

The Cat12Segment interface does not accept compressed .nii.gz files. Everything works fine when one inputs an .nii file.

Actual behavior

This is the error I get:

TraitError: Each element of the 'in_files' trait of a CAT12SegmentInputSpec instance must be a pathlike object or string representing an existing file, but a value of '/pandora/home/johannes.wiesner/work/testing/debug_cat12/T1w.nii.gz' <class 'str'> was specified.

which by the way is a little bit misleading because the file exists but the file extension if the problem (but this is another topic I guess).

Expected behavior

Run Segmentation on a .nii.gz file should work because newer versions of CAT12 are able to deal with .nii.gz files. My CAT12 version is CAT12.9 (r2560) inside an spm12/9.14 environment module.

The segmentation module now supports the input of nii.gz files (not possible for the longitudinal pipeline).

How to replicate the behavior

Here's an example script that should reproduce the error:

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
Debug CAT12Segment
"""

import subprocess
from nipype.interfaces.cat12.preprocess import CAT12Segment
from nipype import Node
import os 

###############################################################################
# Preparation: Download an anatomical files from a public available AWS-bucket 
###############################################################################

# download T1w image
command = """
curl \
--create-dirs https://s3.amazonaws.com/openneuro.org/ds004302/sub-02/anat/sub-02_T1w.nii.gz?versionId=93eQ.AwPcMeccJAT3sO9otYv4A_WH3Bj  \
-o ./T1w.nii.gz
"""
subprocess.run(command,shell=True)

# gunzip image to get from .nii.gz to .nii
subprocess.run("gunzip -c ./T1w.nii.gz > T1w.nii",shell=True)

# get the full path to the directory where this script is being executed
script_dir = os.path.dirname(os.path.realpath(__file__))

# get full path to both images (nipype always needs absolut paths to inputs)
image_nii = os.path.join(script_dir,'T1w.nii')
image_nii_gz = os.path.join(script_dir,'T1w.nii.gz')

###############################################################################
# Run CAT12Segment inside a node on one image
###############################################################################

# create node
cat12segment = Node(CAT12Segment(surface_and_thickness_estimation=0,
                                 surface_measures=0),
                    name='cat12segment')

# input image
cat12segment.inputs.in_files = image_nii_gz

# set working directory of node to script directory
cat12segment.base_dir = script_dir

# run the node
cat12segment.run()

Platform details:

{'commit_hash': '%h',
 'commit_source': 'archive substitution',
 'networkx_version': '3.2.1',
 'nibabel_version': '5.2.0',
 'nipype_version': '1.8.6',
 'numpy_version': '1.26.3',
 'pkg_path': '/csp-tools/anaconda3/envs/csp_wiesner_johannes/lib/python3.9/site-packages/nipype',
 'scipy_version': '1.12.0',
 'sys_executable': '/csp-tools/anaconda3/envs/csp_wiesner_johannes/bin/python',
 'sys_platform': 'linux',
 'sys_version': '3.9.18 | packaged by conda-forge | (main, Dec 23 2023, '
                '16:33:10) \n'
                '[GCC 12.3.0]',
 'traits_version': '6.3.2'}

Execution environment

Choose one

My python environment outside container

JohannesWiesner commented 1 month ago

Maybe @mfmachado has an idea?

effigies commented 1 month ago

This is an explicit helper to remind people to put gunzip interfaces if they need: https://github.com/nipy/nipype/blob/4d1352ade7171fd5f55eff62cee4c99a4f9cfed1/nipype/interfaces/spm/base.py#L617-L631

If SPM now allows compressed NIfTI, we could change it to something like:

allow_compressed=LooseVersion(Info.version()) >= LooseVersion('9.14'),

Would probably take some fiddling to get right, and would be good to know what version they added compression support in.

JohannesWiesner commented 1 month ago

@effigies : It might be more complicated than that, because CAT12 is a toolbox inside SPM12. CAT12 for sure supports .nii.gz files in newer versions but I am not sure if SPM12 does (or at least every version of it). What I can confirm is, that if I run the CAT12 version that my nipype package is currently adressing in it's GUI version, I have no problems with the .nii.gz extension. In other words, the problem apparently lies on the nipype side.

effigies commented 1 month ago

Did earlier versions of CAT12 not support compression? If it has always supported compression, then you can just change ImageFileSPM to ImageFile in the input spec of the CAT12 interfaces, for example, here:

https://github.com/nipy/nipype/blob/4d1352ade7171fd5f55eff62cee4c99a4f9cfed1/nipype/interfaces/cat12/preprocess.py#L25-L31

If it might be different, then we could add a cat12.Info class to detect the version and make an ImageFileCAT12 similar to the ImageFileSPM above that compares the CAT12 version. OTOH, that might be too much work, given that people are unlikely to use new nipype and old CAT12, so you could just drop the ImageFileSPM stuff.

JohannesWiesner commented 1 month ago

Pinging the maintainer @ChristianGaser here, he might be able to answer which versions of CAT12 support .nii.gz files and which do not.

JohannesWiesner commented 1 month ago

Did earlier versions of CAT12 not support compression? If it has always supported compression, then you can just change ImageFileSPM to ImageFile in the input spec of the CAT12 interfaces, for example, here:

https://github.com/nipy/nipype/blob/4d1352ade7171fd5f55eff62cee4c99a4f9cfed1/nipype/interfaces/cat12/preprocess.py#L25-L31

If it might be different, then we could add a cat12.Info class to detect the version and make an ImageFileCAT12 similar to the ImageFileSPM above that compares the CAT12 version. OTOH, that might be too much work, given that people are unlikely to use new nipype and old CAT12, so you could just drop the ImageFileSPM stuff.

Sounds good to me! I had a brief try and just replace ImageFileSPM with ImageFile and it seems to work. Ending up with issue https://github.com/nipy/nipype/issues/3654 but that's another topic of course.

My suggestion:

Replace ImageFileSPM with ImageFile in nipype.interfaces.cat12.preprocess AFTER Christian Gaser has responded from which CAT12 version on compressed files are supported. I can send a PR then. But maybe then the minimum version of CAT12 that supports compressed files should also be added to the nipype docs so folks are aware that they cannot use older versions than this minium version.

ChristianGaser commented 1 month ago

Cat12.8 was the first version that supports nii.gz filesBest, ChristianAm 06.06.2024 um 15:29 schrieb Johannes Wiesner @.***>:

Did earlier versions of CAT12 not support compression? If it has always supported compression, then you can just change ImageFileSPM to ImageFile in the input spec of the CAT12 interfaces, for example, here: https://github.com/nipy/nipype/blob/4d1352ade7171fd5f55eff62cee4c99a4f9cfed1/nipype/interfaces/cat12/preprocess.py#L25-L31 If it might be different, then we could add a cat12.Info class to detect the version and make an ImageFileCAT12 similar to the ImageFileSPM above that compares the CAT12 version. OTOH, that might be too much work, given that people are unlikely to use new nipype and old CAT12, so you could just drop the ImageFileSPM stuff.

Sounds good to me! I had a brief try and just replace ImageFileSPM with ImageFile and it seems to work. Ending up with issue #3654 but that's another topic of course. My suggestion: Replace ImageFileSPM with ImageFile in nipype.interfaces.cat12.preprocess AFTER Christian Gaser has responded from which CAT12 version on compressed files are supported. I can send a PR then. But maybe then the minimum version of CAT12 that supports compressed files should also be added to the nipype docs so folks are aware that they cannot use older versions than this minium version.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>

JailanOweda commented 2 weeks ago

@effigies when we replace ImageFileSPM with ImageFile we get the following error:

SPM version: SPM12 Release: 7771
        SPM path: /apps/lib/matlab/spm12/spm.m
        Item 'Volumes', field 'val': Number of matching files (0) less than required (1).

        Standard error:
        MATLAB code threw an exception:
        No executable modules, but still unresolved dependencies or incomplete module inputs.
        File:/apps/lib/matlab/spm12/spm_jobman.m
        Name:/apps/lib/matlab/spm12/spm_jobman.m
        Line:47
        File:./pyscript_cat12segment.m
        Name:fill_run_job
        Line:115
        File:pm_jobman
        Name:pyscript_cat12segment
        Line:472
        File:÷
        Name:>
        Line:
        Return code: 0

We don't really understand where the error is coming from, since there is no such trait as 'Volumes' for the CAT12 interface. Maybe it's refering to the input image? That it couldn't be read?

I don't know if this information helps you, but we are currently setting all atlases for ROI processing (neuromorphometrics, etc.) to False, as a workaround for the issue #3654.

Here's the full pyscript_cat12segment.m file:

jobs{1}.spm.tools.cat.estwrite.nproc = 32;
jobs{1}.spm.tools.cat.estwrite.opts.affreg = 'mni';
jobs{1}.spm.tools.cat.estwrite.opts.biasacc = 0.5;
jobs{1}.spm.tools.cat.estwrite.extopts.APP = 1070;
jobs{1}.spm.tools.cat.estwrite.extopts.LASstr = 0.5;
jobs{1}.spm.tools.cat.estwrite.extopts.gcutstr = 2.0;
jobs{1}.spm.tools.cat.estwrite.extopts.WMHC = 2;
jobs{1}.spm.tools.cat.estwrite.extopts.vox = 1.5;
jobs{1}.spm.tools.cat.estwrite.extopts.restypes.optimal(1) = 1.0;
jobs{1}.spm.tools.cat.estwrite.extopts.restypes.optimal(2) = 0.1;
jobs{1}.spm.tools.cat.estwrite.extopts.ignoreErrors = 1;
jobs{1}.spm.tools.cat.estwrite.output.surface = 0;
jobs{1}.spm.tools.cat.estwrite.output.ROImenu.atlases.neuromorphometrics = 0;
jobs{1}.spm.tools.cat.estwrite.output.ROImenu.atlases.lpba40 = 0;
jobs{1}.spm.tools.cat.estwrite.output.ROImenu.atlases.hammers = 0;
jobs{1}.spm.tools.cat.estwrite.output.ROImenu.atlases.cobra = 0;
jobs{1}.spm.tools.cat.estwrite.output.ROImenu.atlases.thalamus = 0;
jobs{1}.spm.tools.cat.estwrite.output.ROImenu.atlases.thalamic_nuclei = 0;
jobs{1}.spm.tools.cat.estwrite.output.ROImenu.atlases.suit = 0;
jobs{1}.spm.tools.cat.estwrite.output.GM.native = 0;
jobs{1}.spm.tools.cat.estwrite.output.GM.mod = 1;
jobs{1}.spm.tools.cat.estwrite.output.GM.dartel = 0;
jobs{1}.spm.tools.cat.estwrite.output.WM.native = 0;
jobs{1}.spm.tools.cat.estwrite.output.WM.mod = 1;
jobs{1}.spm.tools.cat.estwrite.output.WM.dartel = 0;
jobs{1}.spm.tools.cat.estwrite.output.CSF.native = 0;
jobs{1}.spm.tools.cat.estwrite.output.CSF.mod = 0;
jobs{1}.spm.tools.cat.estwrite.output.CSF.dartel = 0;
jobs{1}.spm.tools.cat.estwrite.output.label.native = 1;
jobs{1}.spm.tools.cat.estwrite.output.label.warped = 0;
jobs{1}.spm.tools.cat.estwrite.output.label.dartel = 0;
jobs{1}.spm.tools.cat.estwrite.output.labelnative = 1;
jobs{1}.spm.tools.cat.estwrite.output.bias.warped = 1;
jobs{1}.spm.tools.cat.estwrite.output.las.native = 0;
jobs{1}.spm.tools.cat.estwrite.output.las.warped = 0;
jobs{1}.spm.tools.cat.estwrite.output.las.dartel = 0;
jobs{1}.spm.tools.cat.estwrite.output.jacobianwarped = 0;
jobs{1}.spm.tools.cat.estwrite.output.warps(1) = 1;
jobs{1}.spm.tools.cat.estwrite.output.warps(2) = 0;

JailanOweda commented 2 weeks ago

@effigies the problem has something to do with the fact that for some reason when I choose the zipped file as an input it is written like this in the matlab script:

jobs{1}.spm.tools.cat.estwrite.data = {...
'./testing/test_cat12/cat12segment/T1w.nii.gz,1';...
};

So it's looking for the first slice in the zipped file (.gz,1), although if I try choosing the same file in the GUI slices are not displayed for the zipped files, you just take the zipped file as a whole and the script looks as follows:

matlabbatch{1}.spm.tools.cat.estwrite.data = {'./testing/test_cat12/T1w.nii.gz'};

I ran the same script generated by Nipype directly in Matlab before and after removing ,1 from the input path. After removing it, there were no more errors, so the problem is really there. Any ideas on what is causing Nipype to search for slices in the zipped file and how I can avoid that?

nipy / nipype