nipreps / fmriprep

fMRIPrep is a robust and easy-to-use pipeline for preprocessing of diverse fMRI data. The transparent workflow dispenses of manual intervention, thereby ensuring the reproducibility of the results.
https://fmriprep.org
Apache License 2.0
630 stars 292 forks source link

Sanitize long paths in LTA files #1268

Closed romainVala closed 6 years ago

romainVala commented 6 years ago

Hello

I hope it is the wright place to report the bug : I get systematic Segmentation fault for mri_vol2vol.bin when performing the corregistration of the bold data onto the freesurfer T1.

File  "/network/lustre/iss01/apps/lang/anaconda/fmriprep/lib/python3.6/site-packages/niworkflows/nipype/interfaces/freesurfer/base.py", line 144, in run
    return super(FSCommand, self).run(**inputs)
  File "/network/lustre/iss01/apps/lang/anaconda/fmriprep/lib/python3.6/site-packages/niworkflows/nipype/interfaces/base/core.py", line 520, in run
    runtime = self._run_interface(runtime)
  File "/network/lustre/iss01/apps/lang/anaconda/fmriprep/lib/python3.6/site-packages/niworkflows/interfaces/report_base.py", line 60, in _run_interface
    self._post_run_hook(runtime)
  File "/network/lustre/iss01/apps/lang/anaconda/fmriprep/lib/python3.6/site-packages/niworkflows/interfaces/registration.py", line 263, in _post_run_hook
    res = mri_vol2vol.run()
  File "/network/lustre/iss01/apps/lang/anaconda/fmriprep/lib/python3.6/site-packages/niworkflows/nipype/interfaces/freesurfer/base.py", line 144, in run
    return super(FSCommand, self).run(**inputs)
  File "/network/lustre/iss01/apps/lang/anaconda/fmriprep/lib/python3.6/site-packages/niworkflows/nipype/interfaces/base/core.py", line 520, in run
    runtime = self._run_interface(runtime)
  File "/network/lustre/iss01/apps/lang/anaconda/fmriprep/lib/python3.6/site-packages/niworkflows/nipype/interfaces/base/core.py", line 1020, in _run_interface
    self.raise_exception(runtime)
  File "/network/lustre/iss01/apps/lang/anaconda/fmriprep/lib/python3.6/site-packages/niworkflows/nipype/interfaces/base/core.py", line 957, in raise_exception
    ).format(**runtime.dictcopy()))
RuntimeError: Command:
mri_vol2vol --interp nearest --lta /network/lustre/iss01/cenir/analyse/irm/users/romain.valabregue/new/AGENT_10/job/fmriprep/work/fmriprep_wf/single_subject_063_wf/func_preproc_ses_S1_task_rest_echo_3_wf/bold_reg_wf/bbreg_wf/_bold_file_..network..lustre..iss01..cenir..analyse..irm..users..romain.valabregue..new..AGENT_10..bids..sub-063..ses-S1..func..sub-063_ses-S1_task-rest_echo-1_bold.nii.gz/mri_coreg/registration.lta --mov /network/lustre/iss01/cenir/analyse/irm/users/romain.valabregue/new/AGENT_10/job/fmriprep/work/fmriprep_wf/single_subject_063_wf/func_preproc_ses_S1_task_rest_echo_3_wf/bold_reg_wf/bbreg_wf/_bold_file_..network..lustre..iss01..cenir..analyse..irm..users..romain.valabregue..new..AGENT_10..bids..sub-063..ses-S1..func..sub-063_ses-S1_task-rest_echo-1_bold.nii.gz/mri_coreg/uni_xform_masked.nii.gz --targ /network/lustre/iss01/cenir/analyse/irm/users/romain.valabregue/new/AGENT_10/fmriprep/freesurfer/sub-063/mri/brainmask.mgz --o /network/lustre/iss01/cenir/analyse/irm/users/romain.valabregue/new/AGENT_10/job/fmriprep/work/fmriprep_wf/single_subject_063_wf/func_preproc_ses_S1_task_rest_echo_3_wf/bold_reg_wf/bbreg_wf/_bold_file_..network..lustre..iss01..cenir..analyse..irm..users..romain.valabregue..new..AGENT_10..bids..sub-063..ses-S1..func..sub-063_ses-S1_task-rest_echo-1_bold.nii.gz/mri_coreg/uni_xform_masked_warped.nii.gz
Standard output:

Standard error:
/network/lustre/iss01/apps/software/noarch/freesurfer/freesurfer_6.0.0/bin/mri_vol2vol: line 3: 24556 Segmentation fault      mri_vol2vol.bin "$@"
Return code:  139

I then notice that by just changing the content of registration.lta, the exact same mri_vol2vol command would work :

Here is my registration file :

#transform file /network/lustre/iss01/cenir/analyse/irm/users/romain.valabregue/new/AGENT_10/job/fmriprep/work/fmriprep_wf/single_subject_063_wf/func_preproc_ses_S1_task_rest_echo_3_wf/bold_reg_wf/bbreg_wf/_boldfile..network..lustre..iss01..cenir..analyse..irm..users..romain.valabregue..new..AGENT_10..bids..sub-063..ses-S1..func..sub-063_ses-S1_task-rest_echo-1_bold.nii.gz/mri_coreg/registration.lta # created by romain.valabregue on Fri Aug 31 01:38:45 2018

type = 0 # LINEAR_VOX_TO_VOX nxforms = 1 mean = 0.0000 0.0000 0.0000 sigma = 10000.0000 1 4 4 3.060813903808594e+00 8.123503066599369e-03 -2.292764559388161e-02 2.709380531311035e+01 -2.238672226667404e-02 4.111277312040329e-02 -2.914851188659668e+00 1.604636535644531e+02 -7.540457881987095e-03 2.959091663360596e+00 4.096296057105064e-02 2.045415687561035e+01 0.000000000000000e+00 0.000000000000000e+00 0.000000000000000e+00 1.000000000000000e+00 src volume info valid = 1 # volume info valid filename = /network/lustre/iss01/cenir/analyse/irm/users/romain.valabregue/new/AGENT_10/job/fmriprep/work/fmriprep_wf/single_subject_063_wf/func_preproc_ses_S1_task_rest_echo_3_wf/bold_reg_wf/bbreg_wf/_boldfile..network..lustre..iss01..cenir..analyse..irm..users..romain.valabregue..new..AGENT_10..bids..sub-063..ses-S1..func..sub-063_ses-S1_task-rest_echo-1_bold.nii.gz/mri_coreg/uni_xform_masked.nii.gz volume = 66 66 46 voxelsize = 3.030303001403809e+00 3.030303001403809e+00 3.000000000000000e+00 xras = -1.000000000000000e+00 0.000000000000000e+00 0.000000000000000e+00 yras = 0.000000000000000e+00 9.999939203262329e-01 3.490651492029428e-03 zras = 0.000000000000000e+00 -3.490651492029428e-03 9.999939203262329e-01 cras = 0.000000000000000e+00 1.747718811035156e+01 2.381585693359375e+01 dst volume info valid = 1 # volume info valid filename = /network/lustre/iss01/cenir/analyse/irm/users/romain.valabregue/new/AGENT_10/fmriprep/freesurfer/sub-063/mri/brainmask.mgz volume = 256 256 256 voxelsize = 1.000000000000000e+00 1.000000000000000e+00 1.000000000000000e+00 xras = -1.000000000000000e+00 0.000000000000000e+00 0.000000000000000e+00 yras = 0.000000000000000e+00 0.000000000000000e+00 -1.000000000000000e+00 zras = 0.000000000000000e+00 1.000000000000000e+00 0.000000000000000e+00 cras = 5.000000000000000e-01 2.550469207763672e+01 -1.281880187988281e+01 subject sub-063

The problem comes from

filename = /network/lustre/iss01/cenir/analyse/irm/users/romain.valabregue/new/AGENT_10/job/fmriprep/work/fmriprep_wf/single_subject_063_wf/func_preproc_ses_S1_task_rest_echo_3_wf/bold_reg_wf/bbreg_wf/_boldfile..network..lustre..iss01..cenir..analyse..irm..users..romain.valabregue..new..AGENT_10..bids..sub-063..ses-S1..func..sub-063_ses-S1_task-rest_echo-1_bold.nii.gz/mri_coreg/uni_xform_masked.nii.gz

if I then change it to

filename = AGENT_10/job/fmriprep/work/fmriprep_wf/single_subject_063_wf/func_preproc_ses_S1_task_rest_echo_3_wf/bold_reg_wf/bbreg_wf/_boldfile..network..lustre..iss01..cenir..analyse..irm..users..romain.valabregue..new..AGENT_10..bids..sub-063..ses-S1..func..sub-063_ses-S1_task-rest_echo-1_bold.nii.gz/mri_coreg/uni_xform_masked.nii.gz

then it just works fine

note that the filename path is no more correct, but just a little shorter. mri_vol2vol result look fine (it is well corregister to freesufer brain_mask and reslice at the same resolution)

So a solution would be to change the absolut path by just the filename I hope it can be done easily

Regards

Romain

effigies commented 6 years ago

Thanks for the report. We will indeed need to work around this, but it would be worth notifying Freesurfer, as well. Their issues page is not very active, so you may prefer to email the mailing list.

effigies commented 6 years ago

Would you be interested in submitting a pull request to fix this issue?

I think it could be done with a fairly simple interface that we place after the mri_coreg/bbregister nodes. I'd be glad to help you through the process.

oesteban commented 6 years ago

This is somewhat related to #768, isn't it?

effigies commented 6 years ago

Oh yes. I knew we'd done something similar, but hadn't realized how similar. So we could patch BBRegister and MRICoreg interfaces with the same adjustment.

romainVala commented 6 years ago

If you do not have a quick solution I can give a try ... but I am not familiar with nipype, if you could guide me where to look, or how can I see the changes you made for the similar problem?

effigies commented 6 years ago

The fix was here: https://github.com/poldracklab/fmriprep/pull/778/files

The current version of that fix can be found here:

https://github.com/poldracklab/fmriprep/blob/510f28db4aab8a6adde0ccadeba2da7d78ed696e/fmriprep/interfaces/freesurfer.py#L193-L222

So a fix for this would look like:

1) Create PatchedBBRegister and PatchedMRICoreg classes, subclassing from fs.BBRegister and fs.MRICoreg, respectively.

2) Find the out_lta (or appropriately named output file) in a _list_outputs(), and filter it for lines with >=255 characters, rewriting the file, if needed.

If you're interested, feel free to open a pull request at any time, and ask any questions you might have.

romainVala commented 6 years ago

Hi thanks I start to have a loock and I am not sure to look at the wrigh files

for fs.BBRegister you refer to the class define in niworkflows-0.4.3-py3.6.egg/niworkflows/interfaces/registration.py class BBRegisterRPT

if yes I will need to make a pull request also in https://github.com/poldracklab/niworkflows/ ?

Sory for the confusion

effigies commented 6 years ago

Ah, yes, I'd somewhat forgotten about that aspect of it. I think for now let's stay in fMRIPrep. We can create a mixin like this:

class TruncateLTA(object):
    # Use a tuple in case some object produces multiple transforms
    lta_outputs = ('out_lta_file',)

    def _post_run_hook(self, runtime):
        runtime = super(TruncateLTA, self)._post_run_hook(runtime)
        outputs = self.aggregate_outputs(runtime=runtime)

        for lta_name in self.lta_outputs:
            lta_file = outputs[lta_name]
            if not isdefined(lta_file):
                continue
            ...
            # Copy the bits that do the work from above, using lta_file
            # instead of outputs['out_file']
            ...

        return runtime

This takes advantage of post-run-hooks in Nipype interfaces, which are guaranteed to run after the interface has completed, and before any unused files are cleaned up.

Now instead of writing the same patch for each interface, we can rewrite PatchedConcatenateLTA like so:

class PatchedConcatenateLTA(TruncateLTA, ConcatenateLTA):
    lta_outputs = ['out_file']

(You may need to swap the order of TruncateLTA and ConcatenateLTA to make it work properly, but I think this is right.)

And then you can import BBRegisterRPT and MRICoregRPT from niworkflows, and patch them similarly into PatchedBBRegisterRPT and PatchedMRICoregRPT.

Does this all make sense? Please ask as many questions as you need.

romainVala commented 6 years ago

Ok I made just PatchedBBRegisterRPT and PatchedMRICoregRPT I hope it is fine, at least it works localy I did not put too much comment ... todo ... it is my first pull request, not sure also about the process

I'll see on monday ++

effigies commented 6 years ago

Fixed in #1274.