search

PennLINC / qsiprep

Preprocessing of diffusion MRI
http://qsiprep.readthedocs.io
BSD 3-Clause "New" or "Revised" License
140 stars 58 forks source link

[BUG] Error when merging HCP scans #129

Closed mattcieslak closed 3 years ago

mattcieslak commented 4 years ago

Using the 4 dir98/dir99 AP/PA scans from the HCP protocol. Denoising and harmonization work fine, but an error message appears:


Traceback (most recent call last):
  File "/usr/local/miniconda/bin/qsiprep", line 8, in <module>
    sys.exit(main())
  File "/usr/local/miniconda/lib/python3.7/site-packages/qsiprep/cli/run.py", line 566, in main
    qsiprep_wf.run(**plugin_settings)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/workflows.py", line 632, in run
    runner.run(execgraph, updatehash=updatehash, config=self.config)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/plugins/base.py", line 164, in run
    self._clean_queue(jobid, graph, result=result)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/plugins/base.py", line 227, in _clean_queue
    raise RuntimeError("".join(result["traceback"]))
RuntimeError: Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/plugins/multiproc.py", line 67, in run_node
    result["result"] = node.run(updatehash=updatehash)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 516, in run
    result = self._run_interface(execute=True)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 635, in _run_interface
    return self._run_command(execute)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 741, in _run_command
    result = self._interface.run(cwd=outdir)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/base/core.py", line 397, in run
    runtime = self._run_interface(runtime)
  File "/usr/local/miniconda/lib/python3.7/site-packages/qsiprep/interfaces/images.py", line 120, in _run_interface
    plus_b0_images = self.inputs.b0_images_plus
AttributeError: 'ConcatRPESplitsInputSpec' object has no attribute 'b0_images_plus'
mattcieslak commented 4 years ago

Add an option for full AP/PA paired data. It is currently concatenated by default

mattcieslak commented 4 years ago

Partially addressed by #132

italab03 commented 4 years ago

Thank you for making this wonderful software.

I have paired dir68/dir69 AP/PA multi-shell scans. When I preprocessed the data using qsiprep:0.9.0beta1 with docker container, I faced the following error:

200629-16:27:49,277 nipype.workflow WARNING: [Node] Error on "qsiprep_wf.single_subject_00300301_wf.sub_00300301_final_merge_wf.distortion_merger" (/wd/qsiprep_wf/single_subject_00300301_wf/sub_00300301_final_merge_wf/distortion_merger) 200629-16:27:50,983 nipype.workflow ERROR: Node distortion_merger failed to run on host cb064a6d8111. 200629-16:27:51,8 nipype.workflow ERROR: Saving crash info to /out/qsiprep/sub-00300301/log/20200629-094838_9231d587-a471-4a7a-a32a-2f3a9257c56e/crash-20200629-162751-root-distortion_merger-87492890-543d-4ef7-ae9a-8497068a4206.txt Traceback (most recent call last): File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/plugins/multiproc.py", line 67, in run_node result["result"] = node.run(updatehash=updatehash) File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 516, in run result = self._run_interface(execute=True) File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 635, in _run_interface return self._run_command(execute) File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 741, in _run_command result = self._interface.run(cwd=outdir) File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/base/core.py", line 397, in run runtime = self._run_interface(runtime) File "/usr/local/miniconda/lib/python3.7/site-packages/qsiprep/interfaces/dwi_merge.py", line 60, in _run_interface self.inputs.harmonize_b0_intensities) File "/usr/local/miniconda/lib/python3.7/site-packages/qsiprep/interfaces/dwi_merge.py", line 413, in harmonize_b0s b0_mean = index_img(dwi_nii, b0_indices).get_fdata().mean() File "/usr/local/miniconda/lib/python3.7/site-packages/nilearn/image/image.py", line 642, in index_img imgs = check_niimg_4d(imgs) File "/usr/local/miniconda/lib/python3.7/site-packages/nilearn/_utils/niimg_conversions.py", line 356, in check_niimg_4d dtype=dtype) File "/usr/local/miniconda/lib/python3.7/site-packages/nilearn/_utils/niimg_conversions.py", line 276, in check_niimg raise DimensionError(len(niimg.shape), ensure_ndim) nilearn._utils.exceptions.DimensionError: Input data has incompatible dimensionality: Expected dimension is 4D and you provided a 3D image. See http://nilearn.github.io/manipulating_images/input_output.html.

Could you tell me how to solve it.

The command used in the analysis is

docker run -ti --rm -v /mnt/ocean/sourcedata:/data:ro \ -v /home/test/OCEAN/derivatives/Preprocessed:/out \ -v /home/test/OCEAN/derivatives/Preprocessed:/wd \ -v /usr/local/freesurfer/license.txt:/license.txt:ro \ pennbbl/qsiprep:0.9.0beta1 /data /out participant \ --participant_label 00300301 \ --fs-license-file /license.txt \ --hmc_model eddy \ --output-resolution 1.7 --output-space T1w \ --force-spatial-normalization --write-graph \ -v --b0-threshold 100 \ -w /wd \ --combine-all-dwis --distortion-group-merge concat --unringing-method mrdegibbs

` Thank you in advance.

mattcieslak commented 4 years ago

I've seen this one, it will be fixed in 0.9.0beta2. If you switch to "--distortion-group-merge average" it will mimic what the HCP diffusion pipeline does and should finish without any errors. If you're sure you'd rather concat, look out for beta2. Sorry about that!

bwinsto2 commented 4 years ago

hi @mattcieslak - I'm experiencing a similar issue with HCP unprocessed data that have been BIDSified. Perhaps you could help out. I get an error on rpe_concat despite using --distortion-group-merge average

Command:

singularity run --cleanenv -B /hpc900:/hpc900 qsiprep-0.11.0.sif /hpc900/test_data /hpc900/bwinston/qsiprep/output participant --work_dir/hpc900/bwinston/qsiprep/work_dir --fs-license-file /hpc900/bwinston/license.txt --output-resolution 1.25 --output-space T1w --distortion-group-merge average --participant-label 784565

Crash:

Node: qsiprep_wf.single_subject_784565_wf.dwi_preproc_wf.pre_hmc_wf.rpe_concat
Working directory: /hpc900/bwinston/qsiprep/work_dir/qsiprep_wf/single_subject_784565_wf/dwi_preproc_wf/pre_hmc_wf/rpe_concat

Node inputs:

b0_refs = <undefined>
b0_threshold = 100
bids_dwi_files = <undefined>
bval_files = <undefined>
bvec_files = <undefined>
denoising_confounds = <undefined>
dwi_files = <undefined>
harmonize_b0_intensities = True
raw_concatenated_files = <undefined>

Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/plugins/multiproc.py", line 67, in run_node
    result["result"] = node.run(updatehash=updatehash)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 516, in run
    result = self._run_interface(execute=True)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 635, in _run_interface
    return self._run_command(execute)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 741, in _run_command
    result = self._interface.run(cwd=outdir)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/base/core.py", line 397, in run
    runtime = self._run_interface(runtime)
  File "/usr/local/miniconda/lib/python3.7/site-packages/qsiprep/interfaces/dwi_merge.py", line 120, in _run_interface
    pos_merged_nii = math_img('np.clip(img, 0, None)', img=merged_nii)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nilearn/image/image.py", line 887, in math_img
    result = eval(formula, data_dict)
  File "<string>", line 1, in <module>
  File "/usr/local/miniconda/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 1818, in clip
    return _wrapfunc(a, 'clip', a_min, a_max, out=out)
  File "/usr/local/miniconda/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 51, in _wrapfunc
    return getattr(obj, method)(*args, **kwds)
MemoryError: Input formula couldn't be processed, you provided 'np.clip(img, 0, None)',

Is the --distortion-group-merge average not registering? Perhaps there is an issue that I'm using qsiprep version 0.11.0 (I don't have permissions to update, otherwise I would)? Any help is appreciated :)

mattcieslak commented 4 years ago

@bwinsto2 is there any way I could test on your data? I've got it working on HCP data locally so maybe it's BIDS-related

bwinsto2 commented 4 years ago

@mattcieslak thanks for the kind offer - I have been trying to troubleshoot this for the past few days. Ultimately, I fixed the above issue, but I am running into a new error. I think it is BIDS related, as you say. EDDY fails, and part of the message is AcqPara::AcqPara: Unrealistic read-out time. I'm assuming this applies to the readout time for the DWI images and not fieldmaps? Our HCP DWI json files have "TotalReadoutTime" = 0.6. According to this, EDDY accepts readouts of 10-200ms. According to this, BIDS "TotalReadoutTime" should be the same as FSL's readout time, meaning there may be an issue with our dataset since 600ms is way outside the accepted range.

Since you have it working locally, what do your HCP jsons look like? One example of ours is:

"EffectiveEchoSpacing": 0.00078,
"TotalReadoutTime": 0.6,
"PhaseEncodingDirection": "i-",
"EchoTime": 0.0895

I just tested it after calculating readout time manually (around 111ms), and I'm getting the same error. First this happens:

"qsiprep_wf.single_subject_784565_wf.dwi_preproc_wf.hmc_sdc_wf.pre_eddy_b0_ref_wf.enhance_and_mask_b0" in "hpc900/bwinston/qsiprep/work_dir/qsiprep_wf/single_subject_784565_wf/dwi_preproc_wf/hmc_sdc_wf/pre_eddy_b0_ref_wf/enhance_and_mask_b0".
201023-14:07:43,951 nipype.workflow INFO:
     [Node] Running "enhance_and_mask_b0" ("qsiprep.interfaces.nilearn.EnhanceAndSkullstripB0")
 file none does not exist . 
201023-14:08:04,600 nipype.interface WARNING:
     Degenerate Mask case. Using compute_epi_mask
201023-14:08:14,596 nipype.interface WARNING:
     Masking appears to have failed. Using a backup method
 file none does not exist . 
201023-14:08:26,852 nipype.workflow INFO:
     [Node] Finished 
"qsiprep_wf.single_subject_784565_wf.dwi_preproc_wf.hmc_sdc_wf.pre_eddy_b0_ref_wf.enhance_and_mask_b0".

then this:

"qsiprep_wf.single_subject_784565_wf.dwi_preproc_wf.hmc_sdc_wf.b0_ref_brain_to_lps".
201023-14:08:31,865 nipype.workflow INFO:
     [Node] Setting-up "qsiprep_wf.single_subject_784565_wf.dwi_preproc_wf.b0_anat_coreg.b0_to_anat" in "hpc900/bwinston/qsiprep/work_dir/qsiprep_wf/single_subject_784565_wf/dwi_preproc_wf/b0_anat_coreg/b0_to_anat".
201023-14:08:32,12 nipype.workflow INFO:
     [Node] Running "b0_to_anat" ("qsiprep.interfaces.niworkflows.ANTSRegistrationRPT"), a CommandLine Interface with command:
antsRegistration --collapse-output-transforms 1 --dimensionality 3 --initial-moving-transform [ hpc900/bwinston/qsiprep/work_dir/qsiprep_wf/single_subject_784565_wf/anat_preproc_wf/skullstrip_wf/rigid_acpc_resample_brain/highres001_BrainExtractionBrain_trans.nii.gz, hpc900/bwinston/qsiprep/work_dir/qsiprep_wf/single_subject_784565_wf/dwi_preproc_wf/hmc_sdc_wf/b0_ref_to_lps/topup_imain_corrected_avg_unbiasedsharpened_LPS.nii.gz, 0 ] --initialize-transforms-per-stage 0 --interpolation HammingWindowedSinc --output [ transform, transform_Warped.nii.gz ] --transform Rigid[ 0.2 ] --metric Mattes[ hpc900/bwinston/qsiprep/work_dir/qsiprep_wf/single_subject_784565_wf/anat_preproc_wf/skullstrip_wf/rigid_acpc_resample_brain/highres001_BrainExtractionBrain_trans.nii.gz, hpc900/bwinston/qsiprep/work_dir/qsiprep_wf/single_subject_784565_wf/dwi_preproc_wf/hmc_sdc_wf/b0_ref_to_lps/topup_imain_corrected_avg_unbiasedsharpened_LPS.nii.gz, 1, 32, Random, 0.25 ] --convergence [ 10000x1000x10000x10000, 1e-06, 10 ] --smoothing-sigmas 7.0x3.0x1.0x0.0vox --shrink-factors 8x4x2x1 --use-histogram-matching 1 --winsorize-image-intensities [ 0.025, 0.975 ]  --write-composite-transform 0
201023-14:09:37,75 nipype.interface WARNING:
     Error decoding string: 'utf-8' codec can't decode byte 0x90 in position 157: invalid start byte
201023-14:09:37,221 nipype.workflow WARNING:
     Storing result file without outputs
201023-14:09:37,261 nipype.workflow WARNING:
     [Node] Error on "qsiprep_wf.single_subject_784565_wf.dwi_preproc_wf.hmc_sdc_wf.eddy" (hpc900/bwinston/qsiprep/work_dir/qsiprep_wf/single_subject_784565_wf/dwi_preproc_wf/hmc_sdc_wf/eddy)
201023-14:09:37,818 nipype.workflow ERROR:
     Node eddy failed to run on host compute-115.cm.cluster.
201023-14:09:38,22 nipype.workflow ERROR:
     Saving crash info to hpc900/bwinston/qsiprep/output/qsiprep/sub-784565/log/20201022-204159_a3f45ca4-cbfa-46fa-a454-616d8e8440a2/crash-20201023-140937-bwinston-eddy-4ab427ed-5b76-481c-9ba4-373fbf1021f5.txt
Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/plugins/multiproc.py", line 67, in run_node
    result["result"] = node.run(updatehash=updatehash)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 516, in run
    result = self._run_interface(execute=True)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 635, in _run_interface
    return self._run_command(execute)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 741, in _run_command
    result = self._interface.run(cwd=outdir)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/base/core.py", line 397, in run
    runtime = self._run_interface(runtime)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/fsl/epi.py", line 996, in _run_interface
    runtime = super(Eddy, self)._run_interface(runtime)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/base/core.py", line 792, in _run_interface
    self.raise_exception(runtime)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/base/core.py", line 723, in raise_exception
    ).format(**runtime.dictcopy())
RuntimeError: Command:
eddy_openmp  --cnr_maps --field=hpc900/bwinston/qsiprep/work_dir/qsiprep_wf/single_subject_784565_wf/dwi_preproc_wf/hmc_sdc_wf/topup/fieldmap_HZ --flm=linear --ff=10.0 --acqp=hpc900/bwinston/qsiprep/work_dir/qsiprep_wf/single_subject_784565_wf/dwi_preproc_wf/hmc_sdc_wf/gather_inputs/eddy_acqp.txt --bvals=hpc900/bwinston/qsiprep/work_dir/qsiprep_wf/single_subject_784565_wf/dwi_preproc_wf/pre_hmc_wf/rpe_concat/merge__merged.bval --bvecs=hpc900/bwinston/qsiprep/work_dir/qsiprep_wf/single_subject_784565_wf/dwi_preproc_wf/pre_hmc_wf/rpe_concat/merge__merged.bvec --imain=hpc900/bwinston/qsiprep/work_dir/qsiprep_wf/single_subject_784565_wf/dwi_preproc_wf/pre_hmc_wf/rpe_concat/merge__merged.nii.gz --index=hpc900/bwinston/qsiprep/work_dir/qsiprep_wf/single_subject_784565_wf/dwi_preproc_wf/hmc_sdc_wf/gather_inputs/eddy_index.txt --mask=hpc900/bwinston/qsiprep/work_dir/qsiprep_wf/single_subject_784565_wf/dwi_preproc_wf/hmc_sdc_wf/pre_eddy_b0_ref_wf/enhance_and_mask_b0/topup_imain_corrected_avg_mask.nii.gz --interp=spline --data_is_shelled --resamp=jac --niter=5 --nvoxhp=1000 --out=hpc900/bwinston/qsiprep/work_dir/qsiprep_wf/single_subject_784565_wf/dwi_preproc_wf/hmc_sdc_wf/eddy/eddy_corrected --repol --slm=linear
Standard output:
 g> AcqPara::AcqPara: Unrealistic read-out time
EDDY:::  EddyHelperClasses.cpp:::  EDDY::AcqPara::AcqPara(const ColumnVector&, double):  Exception throw�
EDDY::: Eddy failed with message EDDY:::  ECScanClasses.cpp:::  EDDY::ECScanManager::ECScanManager(const string&, const string&, const string&, const string&, const string&, const string&, const string&, const string&, EDDY::ECModel, EDDY::ECModel, const std::vector<unsigned int>&, const EDDY::PolationPara&, EDDY::MultiBandGroups, bool):  Exception thrown
Standard error:

Return code: 1

Thanks so much for any advice.

mattcieslak commented 4 years ago

Did you use a new working directory with the new parameters? It's possible it's still picking up a cached version with your old totalreadouttime

bwinsto2 commented 3 years ago

Indeed. Doing a fresh run after fixing the TotalReadoutTime fixed this particular issue. Thank you!