DIPY reconstruction gives errors for the dipy_3dshore reconstruction workflow

[akelkar26]

Hi, I am running the dipy_3dshore reconstruction workflow for my DSI diffusion data. I had used the 3dshore head motion correction for the preprocessing. I am giving that folder as input. The reconstruction workflow seems to re-compute the preprocessing correctly but throws an error when starting the reconstruction workflow.

Here is my command:

screen docker run -ti \ -v /Users/neurouser1/Documents/TNI/TNI_bids_directory/bids_directory/:/data \ -v /Users/neurouser1/Documents/TNI/qsiprep_dipy_3dshore/:/out \ -v /Users/neurouser1/Documents/fs_files/:/license \ -v /Users/neurouser1/Documents/TNI/scratch/:/scratch \ pennbbl/qsiprep:latest \ /data /out participant \ --participant_label sub-TNI009 \ --output-resolution 1.3 \ --n_cpus ‘15’ \ --omp-nthreads 8 \ --mem_mb 250000 \ --fs-license-file /license/license.txt \ -w /scratch/ \ --recon_input /Users/neurouser1/Documents/TNI/qsiprep_out \ --recon_spec dipy_3dshore \

Here is the output error: Running qsirecon version 0.13.0RC1:

• BIDS dataset path: /data.
• Participant list: ['TNI009'].
• Run identifier: 20210327-114311_22566457-c75a-4b02-ab86-4eb325ba9567.

210327-11:43:20,691 nipype.workflow INFO: found ['/out/qsiprep/sub-TNI009/ses-20170316113937/dwi/sub-TNI009_ses-20170316113937_space-T1w_desc-preproc_dwi.nii.gz'] in /out/qsiprep Process Process-19: Traceback (most recent call last): File "/usr/local/miniconda/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap self.run() File "/usr/local/miniconda/lib/python3.7/multiprocessing/process.py", line 99, in run self._target(*self._args, self._kwargs) File "/usr/local/miniconda/lib/python3.7/site-packages/qsiprep/cli/run.py", line 1114, in build_recon_workflow sloppy=opts.sloppy File "/usr/local/miniconda/lib/python3.7/site-packages/qsiprep/workflows/recon/base.py", line 95, in init_qsirecon_wf sloppy=sloppy File "/usr/local/miniconda/lib/python3.7/site-packages/qsiprep/workflows/recon/base.py", line 221, in init_single_subject_wf omp_nthreads=omp_nthreads) File "/usr/local/miniconda/lib/python3.7/site-packages/qsiprep/workflows/recon/build_workflow.py", line 132, in init_dwi_recon_workflow node_spec) File "/usr/local/miniconda/lib/python3.7/site-packages/qsiprep/workflows/recon/build_workflow.py", line 235, in workflow_from_spec return init_dsi_studio_connectivity_wf(omp_nthreads, has_transform, kwargs) File "/usr/local/miniconda/lib/python3.7/site-packages/qsiprep/workflows/recon/dsi_studio.py", line 267, in init_dsi_studio_connectivity_wf calc_connectivity = pe.Node(DSIStudioAtlasGraph(nthreads=omp_nthreads, params), File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/base/core.py", line 648, in init from_file=from_file, resource_monitor=resource_monitor, inputs File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/base/core.py", line 191, in init self.inputs.trait_set(**inputs) File "/usr/local/miniconda/lib/python3.7/site-packages/traits/has_traits.py", line 1543, in trait_set setattr( self, name, value ) traits.trait_errors.TraitError: Cannot set the undefined 'output_trk' attribute of a 'DSIStudioAtlasGraphInputSpec' object.

Could you please help with the issue? Thank you in advance.

[mattcieslak]

Hi @akelkar26, thanks for reporting. Could you try writing a json file containing:

{ "description": "Reconstruct dwis using Dipy, then convert to mrtrix and dsi studio formats",
  "space": "T1w",
  "name": "convert_shore_recons",
  "atlases": ["schaefer100x7", "schaefer100x17", "schaefer200x7", "schaefer200x17", "schaefer400x7", "schaefer400x17", "brainnetome246", "aicha384", "gordon333", "aal116", "power264"],
  "nodes": [
    {
      "name": "csdsi_3dshore",
      "software": "Dipy",
      "action": "3dSHORE_reconstruction",
      "input": "qsiprep",
      "output_suffix": "3dSHORE",
      "parameters": {
        "radial_order": 6,
        "zeta": 700,
        "tau": 0.025330295910584444,
        "regularization": "L2",
        "lambdaN": 1e-8,
        "lambdaL": 1e-8,
        "write_mif": true,
        "write_fibgz": true
      }
    },
    {
      "name": "streamline_connectivity",
      "software": "DSI Studio",
      "action": "connectivity",
      "input": "csdsi_3dshore",
      "parameters": {
        "turning_angle": 35,
        "method": 0,
        "smoothing": 0.0,
        "step_size": 1.0,
        "min_length": 10,
        "max_length": 250,
        "seed_plan": 0,
        "interpolation": 0,
        "initial_dir": 2,
        "fiber_count": 5000000,
        "connectivity_value": "count,ncount,mean_length,gfa",
        "connectivity_type": "pass,end"
       }
    }
  ]
}

and mounting this in your container and specifying its mounted path as the argument for --recon-spec?

[akelkar26]

Yes, I will do that. I will try running it with the json file and let you know how it goes. Thank you

[akelkar26]

Hi, I created the json file and gave it as the path to recon-spec

Here is my command: screen docker run -ti \ -v /Users/neurouser1/Documents/TNI/TNI_bids_directory/bids_directory/:/data \ -v /Users/neurouser1/Documents/TNI/qsiprep_dipy_3dshore/:/out \ -v /Users/neurouser1/Documents/fs_files/:/license \ -v /Users/neurouser1/Documents/TNI/scratch/:/scratch \ pennbbl/qsiprep:latest \ /data /out participant \ --participant_label sub-TNI009 \ --output-resolution 1.3 \ --n_cpus ‘15’ \ --omp-nthreads 8 \ --mem_mb 250000 \ --fs-license-file /license/license.txt \ -w /scratch/ \ --recon_input /Users/neurouser1/Documents/TNI/qsiprep_out \ --recon_spec /Users/neurouser1/Documents/TNI/qsistudio_dipyrecon.json \

I get this error now, seems like it doesn't find the file:

Running qsirecon version 0.13.0RC1:
  * BIDS dataset path: /data.
  * Participant list: ['TNI009'].
  * Run identifier: 20210406-124154_20891eab-75be-4870-835e-40ad30624736.

Process Process-19: Traceback (most recent call last): File "/usr/local/miniconda/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap self.run() File "/usr/local/miniconda/lib/python3.7/multiprocessing/process.py", line 99, in run self._target(*self._args, **self._kwargs) File "/usr/local/miniconda/lib/python3.7/site-packages/qsiprep/cli/run.py", line 1114, in build_recon_workflow sloppy=opts.sloppy File "/usr/local/miniconda/lib/python3.7/site-packages/qsiprep/workflows/recon/base.py", line 95, in init_qsirecon_wf sloppy=sloppy File "/usr/local/miniconda/lib/python3.7/site-packages/qsiprep/workflows/recon/base.py", line 161, in init_single_subject_wf spec = _load_recon_spec(recon_spec, sloppy=sloppy) File "/usr/local/miniconda/lib/python3.7/site-packages/qsiprep/workflows/recon/base.py", line 235, in _load_recon_spec raise Exception("{} is not a file that exists or in {}".format(spec_name, prepackaged)) Exception: /Users/neurouser1/Documents/TNI/qsistudio_dipyrecon.json is not a file that exists or in ['dipy_3dshore', 'mrtrix_singleshell_ss3t', 'dsi_studio_gqi', 'dipy_mapmri', 'mrtrix_multishell_msmt', 'mrtrix_tckglobal', 'csdsi_3dshore', 'mrtrix_multishell_msmt_noACT', 'mrtrix_singleshell_ss3t_noACT']

Am I doing something wrong? Thanks in advance for your help.

[akelkar26]

Hi, Good morning. Just reaching out to ask if there is any other method I should try to get the dipy reconsturction working. Thank you

[mattcieslak]

Sorry for the delay -- it looks like a volume mounding issue. One nice way around these is to install the qsiprep-container package in your local python distribution.

$ pip install qsiprep-container
$ qsiprep-docker \
--image pennbbl/qsiprep:latest \
/Users/neurouser1/Documents/TNI/TNI_bids_directory/bids_directory \
/Users/neurouser1/Documents/TNI/qsiprep_dipy_3dshore \
participant \
--participant_label sub-TNI009 \
--output-resolution 1.3 \
--n_cpus 15 \
--omp-nthreads 8 \
--mem_mb 250000 \
--fs-license-file /Users/neurouser1/Documents/fs_files/license.txt \
-w /Users/neurouser1/Documents/TNI/scratch \
--recon_input /Users/neurouser1/Documents/TNI/qsiprep_out \
--recon_spec /Users/neurouser1/Documents/TNI/qsistudio_dipyrecon.json

If you can't install to python on your machine, you could alternatively try this docker command

screen docker run -ti
-v /Users/neurouser1/Documents/TNI/TNI_bids_directory/bids_directory/:/data \
-v /Users/neurouser1/Documents/TNI/qsiprep_dipy_3dshore/:/out \
-v /Users/neurouser1/Documents/fs_files/:/license \
-v /Users/neurouser1/Documents/TNI/scratch/:/scratch \
-v /Users/neurouser1/Documents/TNI/:/specs \
-v /Users/neurouser1/Documents/TNI/qsiprep_out:/prep_out \
pennbbl/qsiprep:latest \
/data /out participant \
--participant_label sub-TNI009 \
--output-resolution 1.3 \
--n_cpus 15 \
--omp-nthreads 8 \
--mem_mb 250000 \
--fs-license-file /license/license.txt \
-w /scratch \
--recon_input /prep_out \
--recon_spec /specs/qsistudio_dipyrecon.json

[akelkar26]

Hi,

Thank you so much for the input. The docker command worked well.
The recon pipeline ran. However, the html file doesn't show the connectivity matrices (svg file doesn't show). Also, there is no .mat file (for DSIStudio or MRTrix) which gives the connectivity measures as matrices. Do I need to add anything else to the pre-defined json file to get the figure and the mat files? Thank you once again.

[mattcieslak]

My mistake, there is one more entry needed for a step in the reconstruction workflows to write their outputs. Could you try:

{ "description": "Reconstruct dwis using Dipy, then convert to mrtrix and dsi studio formats",
  "space": "T1w",
  "name": "convert_shore_recons",
  "atlases": ["schaefer100x7", "schaefer100x17", "schaefer200x7", "schaefer200x17", "schaefer400x7", "schaefer400x17", "brainnetome246", "aicha384", "gordon333", "aal116", "power264"],
  "nodes": [
    {
      "name": "csdsi_3dshore",
      "software": "Dipy",
      "action": "3dSHORE_reconstruction",
      "input": "qsiprep",
      "output_suffix": "3dSHORE",
      "parameters": {
        "radial_order": 6,
        "zeta": 700,
        "tau": 0.025330295910584444,
        "regularization": "L2",
        "lambdaN": 1e-8,
        "lambdaL": 1e-8,
        "write_mif": true,
        "write_fibgz": true
      }
    },
    {
      "name": "streamline_connectivity",
      "software": "DSI Studio",
      "action": "connectivity",
      "input": "csdsi_3dshore",
      "output_suffix": "3dSHOREConn",
      "parameters": {
        "turning_angle": 35,
        "method": 0,
        "smoothing": 0.0,
        "step_size": 1.0,
        "min_length": 10,
        "max_length": 250,
        "seed_plan": 0,
        "interpolation": 0,
        "initial_dir": 2,
        "fiber_count": 5000000,
        "connectivity_value": "count,ncount,mean_length,gfa",
        "connectivity_type": "pass,end"
       }
    }
  ]
}

[akelkar26]

Hi,

Thank you so much for the reply. I added the new lines of code and re-ran the docker. There were no errors in the processing (but it ran for only 18 hours for dipy when it generally takes 22 hours for dsistudio). I got a 3dSHOREConn.mat file but the file seems to have data missing. Also, the connectome svg figure is still blank and doesn't show in the html output. As seen in the screenshot below, it only has the region labels and indices for each atlas (no connectivity matrices).

Is there anything else I need to add to get it to work? Thanks so much for helping me troubleshoot this.

[rtybanana]

Hi @mattcieslak,

I'm having the same issue as @akelkar26. First the output_trk error and then this issue of only region labels and ids in the output .mat file. Do you have any further insight on this?

[mattcieslak]

Would anyone be willing to share their preprocessed data? I'm troubleshooting this now for a 0.13.0 release

I replicated this on some test data