traits.trait_errors.TraitError: The 'output_image' trait of a _FixN4BiasFieldCorrectionOutputSpec instance must be a pathlike object or string representing an existing file

[shawnyoung23]

What happened?

hi i was trying to run fMRIprep uisng the code below

fmriprep /inputfolder/ /output/ participant --participant-label sub-Bxxxx --fs-license-file /pathtolicense/license.txt --nthreads 5 --omp-nthreads 5 --mem-mb 15G --use-aroma --ignore slicetiming --ignore t2w

( i have used xxxx to anomymize the info )

this code was proven to be functional on another participant; but it returned the error below after running for 7 hours on this participant;

error:

traits.trait_errors.TraitError: The 'output_image' trait of a _FixN4BiasFieldCorrectionOutputSpec instance must be a pathlike object or string representing an existing file, but a value of '/work/fmriprep_23_0_wf/single_subject_Bxxxx_wf/func_preproc_ses_xxxx_task_rest24mm490scansms8_run_1_echo_1_wf/initial_boldref_wf/enhance_and_skullstrip_bold_wf/n4_correct/sub-Bxxxx_ses-xxxx_task-rest24mm490scansms8_run-1_echo-1_bold_average_corrected.nii.gz' <class 'str'> was specified.

the parameters used was 8 cpu, 32G RAM, 100G disk space; our data has 490 volumes, with TR=0.735, TE being 0.035, dimension being 929264*490

What command did you use?

fmriprep /inputfolder/ /output/ participant --participant-label sub-Bxxxx --fs-license-file /pathtolicense/license.txt --nthreads 5 --omp-nthreads 5 --mem-mb 15G --use-aroma --ignore slicetiming --ignore t2w

What version of fMRIPrep are you running?

the latest version in dockerhub is pulled and used

How are you running fMRIPrep?

Docker

Is your data BIDS valid?

Yes

Are you reusing any previously computed results?

No

Please copy and paste any relevant log output.

�2023-05-12T10:30:18.271679031Z    File "/opt/conda/lib/python3.9/site-packages/nipype/pipeline/engine/nodes.py", line 645, in _run_interface

F2023-05-12T10:30:18.271682869Z      return self._run_command(execute)

�2023-05-12T10:30:18.271686160Z    File "/opt/conda/lib/python3.9/site-packages/nipype/pipeline/engine/nodes.py", line 771, in _run_command

B2023-05-12T10:30:18.271697312Z      raise NodeExecutionError(msg)

�2023-05-12T10:30:18.271700553Z  nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node n4_correct.

!2023-05-12T10:30:18.271703711Z  

)2023-05-12T10:30:18.271708212Z  Cmdline:


2023-05-12T10:30:18.271711698Z      N4BiasFieldCorrection --bspline-fitting [ 200 ] -d 3 --input-image /work/fmriprep_23_0_wf/single_subject_B10203_wf/func_preproc_ses_WQZHEADFMRI20230317133821278000_task_rest24mm490scansms8_run_1_echo_1_wf/initial_boldref_wf/gen_avg/sub-B10203_ses-WQZHEADFMRI20230317133821278000_task-rest24mm490scansms8_run-1_echo-1_bold_average.nii.gz --output sub-B10203_ses-WQZHEADFMRI20230317133821278000_task-rest24mm490scansms8_run-1_echo-1_bold_average_corrected.nii.gz -r --weight-image /work/fmriprep_23_0_wf/single_subject_B10203_wf/func_preproc_ses_WQZHEADFMRI20230317133821278000_task_rest24mm490scansms8_run_1_echo_1_wf/initial_boldref_wf/enhance_and_skullstrip_bold_wf/check_hdr/tpl-MNI152NLin2009cAsym_res-01_label-brain_probseg_trans_hdr.nii.gz

(2023-05-12T10:30:18.271716073Z  Stdout:

!2023-05-12T10:30:18.271718858Z  

(2023-05-12T10:30:18.271721571Z  Stderr:

!2023-05-12T10:30:18.271724433Z  

+2023-05-12T10:30:18.271727169Z  Traceback:

D2023-05-12T10:30:18.271730030Z    Traceback (most recent call last):

�2023-05-12T10:30:18.271732939Z      File "/opt/conda/lib/python3.9/site-packages/nipype/interfaces/base/core.py", line 453, in aggregate_outputs

@2023-05-12T10:30:18.271736107Z        setattr(outputs, key, val)

�2023-05-12T10:30:18.271738939Z      File "/opt/conda/lib/python3.9/site-packages/nipype/interfaces/base/traits_extension.py", line 330, in validate

s2023-05-12T10:30:18.271742018Z        value = super(File, self).validate(objekt, name, value, return_pathlike=True)

�2023-05-12T10:30:18.271745617Z      File "/opt/conda/lib/python3.9/site-packages/nipype/interfaces/base/traits_extension.py", line 135, in validate

J2023-05-12T10:30:18.271748797Z        self.error(objekt, name, str(value))

�2023-05-12T10:30:18.271751840Z      File "/opt/conda/lib/python3.9/site-packages/traits/base_trait_handler.py", line 74, in error

72023-05-12T10:30:18.271754858Z        raise TraitError(

72023-05-12T10:30:18.271757734Z   traits.trait_errors.TraitError: The 'output_image' trait of a _FixN4BiasFieldCorrectionOutputSpec instance must be a pathlike object or string representing an existing file, but a value of '/work/fmriprep_23_0_wf/single_subject_Bxxxx_wf/func_preproc_ses_xxxx_task_rest24mm490scansms8_run_1_echo_1_wf/initial_boldref_wf/enhance_and_skullstrip_bold_wf/n4_correct/sub-Bxxxx_ses-xxxx_task-rest24mm490scansms8_run-1_echo-1_bold_average_corrected.nii.gz' <class 'str'> was specified.

!2023-05-12T10:30:18.271761555Z  

e2023-05-12T10:30:18.271764300Z    During handling of the above exception, another exception occurred:

!2023-05-12T10:30:18.271767368Z  

D2023-05-12T10:30:18.271770152Z    Traceback (most recent call last):

�2023-05-12T10:30:18.271775258Z      File "/opt/conda/lib/python3.9/site-packages/nipype/interfaces/base/core.py", line 398, in run

L2023-05-12T10:30:18.271781446Z        runtime = self._post_run_hook(runtime)

�2023-05-12T10:30:18.271784457Z      File "/opt/conda/lib/python3.9/site-packages/nipype/interfaces/mixins/fixheader.py", line 127, in _post_run_hook

h2023-05-12T10:30:18.271787504Z        outputs = self.aggregate_outputs(runtime=runtime).get_traitsfree()

�2023-05-12T10:30:18.271790496Z      File "/opt/conda/lib/python3.9/site-packages/nipype/interfaces/base/core.py", line 460, in aggregate_outputs

B2023-05-12T10:30:18.271793509Z        raise FileNotFoundError(msg)


�2023-05-12T10:30:18.271796578Z   FileNotFoundError: No such file or directory '/work/fmriprep_23_0_wf/single_subject_Bxxxx_wf/func_preproc_ses_xxxx_task_rest24mm490scansms8_run_1_echo_1_wf/initial_boldref_wf/enhance_and_skullstrip_bold_wf/n4_correct/sub-Bxxxx_ses-xxxx_task-rest24mm490scansms8_run-1_echo-1_bold_average_corrected.nii.gz' for output 'output_image' of a FixN4BiasFieldCorrection interface

!2023-05-12T10:30:18.271800138Z  

!2023-05-12T10:30:18.271802871Z  

?2023-05-12T10:30:24.407066899Z  230512-10:30:24,406 cli ERROR:

�2023-05-12T10:30:24.407098798Z     Preprocessing did not finish successfully. Errors occurred while processing data from participants: B10203 (1). Check the HTML reports for details.

?2023-05-12T10:30:24.407104198Z  230512-10:30:24,406 cli ERROR:

�2023-05-12T10:30:24.407110928Z     Preprocessing did not finish successfully. Errors occurred while processing data from participants: B10203 (1). Check the HTML reports for details.
V2023-05-12T10:30:27.879656462Z  Sentry is attempting to send 5 pending error messages
82023-05-12T10:30:27.879693497Z  Waiting up to 2 seconds
52023-05-12T10:30:27.879698598Z  Press Ctrl-C to quit

Additional information / screenshots

No response

[shawnyoung23]

any suggestions would be greatly appreciated! :)

[effigies]

The TraitError is just https://github.com/nipy/nipype/issues/3493, which makes it almost impossible to debug things.

I would guess this is a memory issue. Please fill in the actual command you used. If you used docker, it should not start with fmriprep.

[shawnyoung23]

hi effigies, thank you for the quick reply!

we are using a platform where every time a job is submitted, a new terminal containing the docker(nipreps) is opened; so actually the code above is the original code...

i will go and have a look at the error you kindly suggested, thanks!

[effigies]

I see. If you have access to the working directory, I would try running N4BiasFieldCorrection --bspline-fitting [ 200 ] -d 3 --input-image /work/fmriprep_23_0_wf/single_subject_B10203_wf/func_preproc_ses_WQZHEADFMRI20230317133821278000_task_rest24mm490scansms8_run_1_echo_1_wf/initial_boldref_wf/gen_avg/sub-B10203_ses-WQZHEADFMRI20230317133821278000_task-rest24mm490scansms8_run-1_echo-1_bold_average.nii.gz --output sub-B10203_ses-WQZHEADFMRI20230317133821278000_task-rest24mm490scansms8_run-1_echo-1_bold_average_corrected.nii.gz -r --weight-image /work/fmriprep_23_0_wf/single_subject_B10203_wf/func_preproc_ses_WQZHEADFMRI20230317133821278000_task_rest24mm490scansms8_run_1_echo_1_wf/initial_boldref_wf/enhance_and_skullstrip_bold_wf/check_hdr/tpl-MNI152NLin2009cAsym_res-01_label-brain_probseg_trans_hdr.nii.gz directly and seeing what happens.

[shawnyoung23]

unfortunately after the job was killed, all the working directory was deleted automatically by the platform by default; i removed '--nthreads 5 --omp-nthreads 5 --mem-mb 15G' , in a hope to free fMRIPREPs from any potential memory limit, and would update the results here once available thank you effigies, in advance

[shawnyoung23]

i ran 3 participants in batch last night, while 2 of these finished as planned, the same problem persisted for the old one; would update if i find a solution

[carolinatravassos]

hi! any update on this issue?

[mgxd]

If you still have your working directory, we might be able to get a better idea of why N4BiasCorrection is failing if you can share the contents of:

<work-dir>/fmriprep_2x_0_wf/single_subject_xxxx_wf/func_preproc_...wf/initial_boldref_wf/enhance_and_skullstrip_bold_wf/n4_correct/_report/report.rst

[carolinatravassos]

Node: single_subject_p0013_wf (func_preproc_ses_01_task_S1Map_run_1_wf (initial_boldref_wf (enhance_and_skullstrip_bold_wf (n4_correct (fixes)
==============================================================================================================================================

 Hierarchy : fmriprep_23_0_wf.single_subject_p0013_wf.func_preproc_ses_01_task_S1Map_run_1_wf.initial_boldref_wf.enhance_and_skullstrip_bold_wf.n4_correct
 Exec ID : n4_correct

Original Inputs
---------------

* args : <undefined>
* bias_image : <undefined>
* bspline_fitting_distance : 200.0
* bspline_order : <undefined>
* convergence_threshold : <undefined>
* copy_header : True
* dimension : 3
* environ : {'NSLOTS': '1'}
* histogram_sharpening : <undefined>
* input_image : /work/fmriprep_23_0_wf/single_subject_p0013_wf/func_preproc_ses_01_task_S1Map_run_1_wf/initial_boldref_wf/gen_avg/sub-p0013_ses-01_task-S1Map_run-1_bold_average.nii.gz
* mask_image : <undefined>
* n_iterations : <undefined>
* num_threads : 1
* output_image : <undefined>
* rescale_intensities : True
* save_bias : False
* shrink_factor : <undefined>
* weight_image : /work/fmriprep_23_0_wf/single_subject_p0013_wf/func_preproc_ses_01_task_S1Map_run_1_wf/initial_boldref_wf/enhance_and_skullstrip_bold_wf/check_hdr/tpl-MNI152NLin2009cAsym_res-01_label-brain_probseg_trans_hdr.nii.gz

[effigies]

The reports don't get fully generated on failure anymore. I think it's related to the failure to https://github.com/nipy/nipype/issues/3493, but I haven't had time to debug. Instead, can you look at n4_correct/command.txt?

If you used fmriprep-docker (which I highly recommend), then you can take your existing command and add --shell to it to get a shell inside the container. From there, you should be able to cd into /work/fmriprep_23_0_wf/single_subject_p0013_wf/func_preproc_ses_01_task_S1Map_run_1_wf/initial_boldref_wf/enhance_and_skullstrip_bold_wf/n4_correct/.

You could then just bash -x command.txt to print and then run the command inside the working directory exactly as fMRIPrep was trying to do.

[shawnyoung23]

hi i have followed your suggestions and obtained the report file in n4_correct/, which i have pasted below:

Node: single_subject_B10203_wf (func_preproc_ses_WQZHEADFMRI20230317133821278000_task_rest24mm490scansms8_run_1_echo_1_wf (initial_boldref_wf (enhance_and_skullstrip_bold_wf (n4_correct (fixes)
=================================================================================================================================================================================================

 Hierarchy : fmriprep_23_0_wf.single_subject_B10203_wf.func_preproc_ses_WQZHEADFMRI20230317133821278000_task_rest24mm490scansms8_run_1_echo_1_wf.initial_boldref_wf.enhance_and_skullstrip_bold_wf.n4_correct
 Exec ID : n4_correct

Original Inputs
---------------

* args : <undefined>
* bias_image : <undefined>
* bspline_fitting_distance : 200.0
* bspline_order : <undefined>
* convergence_threshold : <undefined>
* copy_header : True
* dimension : 3
* environ : {'NSLOTS': '1'}
* histogram_sharpening : <undefined>
* input_image : /tmp/work/fmriprep_23_0_wf/single_subject_B10203_wf/func_preproc_ses_WQZHEADFMRI20230317133821278000_task_rest24mm490scansms8_run_1_echo_1_wf/initial_boldref_wf/gen_avg/sub-B10203_ses-WQZHEADFMRI20230317133821278000_task-rest24mm490scansms8_run-1_echo-1_bold_average.nii.gz
* mask_image : <undefined>
* n_iterations : <undefined>
* num_threads : 1
* output_image : <undefined>
* rescale_intensities : True
* save_bias : False
* shrink_factor : <undefined>
* weight_image : /tmp/work/fmriprep_23_0_wf/single_subject_B10203_wf/func_preproc_ses_WQZHEADFMRI20230317133821278000_task_rest24mm490scansms8_run_1_echo_1_wf/initial_boldref_wf/enhance_and_skullstrip_bold_wf/check_hdr/tpl-MNI152NLin2009cAsym_res-01_label-brain_probseg_trans_hdr.nii.gz

[effigies]

If you're able to run the command in the working directory, we might be able to get usable output. As it is, that is not enough for us to understand what's going wrong.

Are you able to share the files that are pointed to by input_image and weight_image? If so, then perhaps we can try to do this ourselves.

[shawnyoung23]

hi thank you effigies, for the quick response and the guidance throughout. Just want to let you know how much i appreciate that. i ran the lines in command.txt, unfortunately no output was generated. i have attached the relevant files here,

input_image: sub-B10203_ses-WQZHEADFMRI20230317133821278000_task-rest24mm490scansms8_run-1_echo-1_bold_average.nii.gz

weight_image: tpl-MNI152NLin2009cAsym_res-01_label-brain_probseg_trans_hdr.nii.gz

all files in n4_correct folder after i rerun the command n4_correct.zip

please feel free to have a look if you have spare time :)

[shawnyoung23]

btw this is the exact command in command.txt file:

N4BiasFieldCorrection --bspline-fitting '[' 200 ']' -d 3 --input-image /tmp/work/fmriprep_23_0_wf/single_subject_B10203_wf/func_preproc_ses_WQZHEADFMRI20230317133821278000_task_rest24mm490scansms8_run_1_echo_1_wf/initial_boldref_wf/gen_avg/sub-B10203_ses-WQZHEADFMRI20230317133821278000_task-rest24mm490scansms8_run-1_echo-1_bold_average.nii.gz --output sub-B10203_ses-WQZHEADFMRI20230317133821278000_task-rest24mm490scansms8_run-1_echo-1_bold_average_corrected.nii.gz -r --weight-image /tmp/work/fmriprep_23_0_wf/single_subject_B10203_wf/func_preproc_ses_WQZHEADFMRI20230317133821278000_task_rest24mm490scansms8_run_1_echo_1_wf/initial_boldref_wf/enhance_and_skullstrip_bold_wf/check_hdr/tpl-MNI152NLin2009cAsym_res-01_label-brain_probseg_trans_hdr.nii.gz

[alexsayal]

Hello,

I have the same error in a single subject of a dataset.

This is the output of the N4BiasFieldCorrection command with the '-v' option enabled (without this, nothing is printed to the console):

Running N4 for 3-dimensional images.

Mask not read.  Using the entire image as the mask.

Original intensity range:  [0, 12154.3]

***********************************************************
Warning:  Your input image contains nonpositive values
which could cause failure or problematic results.  A
possible workaround would be to:
   1. rescale your image to positive values e.g., [10,100].
   2. run N4 on your rescaled image.
   3. (optional) rescale the N4 output to the original
      intensity range.
***********************************************************

Specified spline distance: 200mm
  original image size:  [76, 76, 42]
  padded image size:  [80, 80, 68]
  number of control points:  [4, 4, 4]

Exception caught: 
itk::ExceptionObject (0x3454110)
Location: "unknown" 
File: /src/ants/build/staging/include/ITK-5.1/itkImageToImageFilter.hxx
Line: 220
Description: itk::ERROR: N4BiasFieldCorrectionImageFilter(0x34567b0): Inputs do not occupy the same physical space! 
InputImage Origin: [-1.0660573e+02, 6.4281643e+01, -2.0323241e+01], InputImageMaskImage Origin: [-1.0660760e+02, 6.4281683e+01, -2.0324781e+01]
        Tolerance: 1.0105263e-05
InputImage Direction: 9.9779676e-01 6.6197840e-02 4.4136691e-03
6.6197840e-02 -9.8895503e-01 -1.3261142e-01
4.4136690e-03 -1.3261142e-01 9.9115828e-01
, InputImageMaskImage Direction: 9.9779606e-01 6.6276418e-02 3.2346698e-03
6.6119201e-02 -9.8895504e-01 -1.3265060e-01
5.5926639e-03 -1.3257212e-01 9.9115758e-01

        Tolerance: 1.0000000e-06

Just for reference this is the full command:

N4BiasFieldCorrection --bspline-fitting [ 200 ] -d 3 --input-image /scratch/fmriprep_23_0_wf/single_subject_07_wf/func_preproc_task_AA_acq_0500_run_1_wf/initial_boldref_wf/gen_avg/sub-07_task-AA_acq-0500_run-1_bold_average.nii.gz --output sub-07_task-AA_acq-0500_run-1_bold_average_corrected.nii.gz -r --weight-image /scratch/fmriprep_23_0_wf/single_subject_07_wf/func_preproc_task_AA_acq_0500_run_1_wf/initial_boldref_wf/enhance_and_skullstrip_bold_wf/check_hdr/tpl-MNI152NLin2009cAsym_res-01_label-brain_probseg_trans_hdr.nii.gz -v

I hope this helps!

[effigies]

Thank you both! That indeed seems very helpful.

[effigies]

Well, this one's fun. This actually works just fine locally, but crashes if I run it inside the latest fMRIPrep image. So it seems to be something that's changed in recent versions of ANTs. I think we should probably prioritize upgrading ANTs for the next release. The alternative is to figure out exactly why the affines aren't quite close enough for ANTs' liking.

[shawnyoung23]

Well, this one's fun. This actually works just fine locally, but crashes if I run it inside the latest fMRIPrep image. So it seems to be something that's changed in recent versions of ANTs. I think we should probably prioritize upgrading ANTs for the next release. The alternative is to figure out exactly why the affines aren't quite close enough for ANTs' liking.

Thank you so much for the info! i would try to download a previously released image to bypass the situation then ---would update the result later~

[effigies]

Oh, sorry. I mean that it's fixed in newer versions of ANTs. We haven't upgraded ANTs in fMRIPrep in a couple years, so using an older image is unlikely to help.

[shawnyoung23]

oh, i see; but thanks anyway :)

[shawnyoung23]

hey effigies, thank you for the commit! just wondering if it is possible that i can pull it from Docker Hub? im really fresh here with Git...

[effigies]

In a bit, you can try pulling nipreps/fmriprep:unstable. That will be at least an hour. We have a test that fails periodically and may need restarting. If that happens it probably won't be available until morning (US EDT).

[shawnyoung23]

:) thanks!

[effigies]

Sorry, thought I posted this morning. The latest :unstable image does contain this patch.

[shawnyoung23]

just want to let you know know much i appreciate this, thank you!

[alexsayal]

I confirm that the N4BiasFieldCorrection is successful using the unstable version.

Thank you!