search

bids-apps / MRtrix3_connectome

Generate subject connectomes from raw BIDS data & perform inter-subject connection density normalisation, using the MRtrix3 software package.
http://www.mrtrix.org/
Apache License 2.0
48 stars 26 forks source link

output-verbosity 3 and tractogram #91

Closed dkp closed 2 years ago

dkp commented 3 years ago

Hi Robert,

participant level ran flawlessly once the derivatives directory was no longer nested in the bids dir. I ran with --output-verbosity 3 which should generate a tractogram.
Version 0.42 generates a monster sub-219_tractogram.tck file in the tractogram directory.

However, version 0.5 does not generate the *.tck file: MRtrix3_connectome-participant/sub-219/ses-itbs/tractogram: sub-219_ses-itbs_space-dwi_tdi.nii.gz

Isn't the tck file the tractogram? Perhaps my knowledge of MRtrix3 is just woefully inadequate, but it seems like a major change.

Thanks for your patience.

Lestropie commented 3 years ago

The tractogram file is certainly flagged to export at output verbosity 3, and the conditions for export are the same as those for the file you report seeing; so I'm not sure exactly how it is that's happened. I'll try to reproduce locally... can you provide the exact conditions in which you're running (i.e. environment & command-line options)?

dkp commented 3 years ago

Thanks Robert,

I'm attaching the job script I ran. The environment is our HPC with CentOS7.

The BIDS directory is called Nifti and contains a single subject (me) that I'm having the students process for the class.

ls Nifti CHANGES code dataset_description.json participants.tsv README sub-219 task-rest_bold.json

I just looked at the job log, and it seems to complain near the end (line 64).

And the tck file does appear to be in the scratch directory:

dkp@login2:mrtrix3_connectome.py-tmp-4MH5OG$ ls -l total 31263744

-rw-r--r-- 1 dkp dkp 49 Nov 16 20:28 5TT.json -rw-r--r-- 1 dkp dkp 63425740 Nov 16 20:28 5TT.mif -rw-r--r-- 1 dkp dkp 516800190 Nov 17 09:55 assignments.csv -rw-r--r-- 1 dkp dkp 1160607 Nov 17 09:54 connectome.csv -rw-r--r-- 1 dkp dkp 61545 Nov 16 20:05 dwi_mask.mif -rw-r--r-- 1 dkp dkp 62328376 Nov 16 20:05 dwi.mif -rw-r--r-- 1 dkp dkp 313 Nov 17 10:28 error.txt -rw-r--r-- 1 dkp dkp 68957516 Nov 17 10:27 exemplars.tck -rw-r--r-- 1 dkp dkp 128549844 Nov 16 20:19 FOD_WM.mif lrwxrwxrwx 1 dkp dkp 34 Nov 16 20:28 fsaverage -> /opt/freesurfer/subjects/fsaverage lrwxrwxrwx 1 dkp dkp 38 Nov 16 20:28 lh.EC_average -> /opt/freesurfer/subjects/lh.EC_average -rw-r--r-- 1 dkp dkp 4981 Nov 17 10:28 log.txt -rw-r--r-- 1 dkp dkp 1154830 Nov 17 09:58 meanlength.csv -rw-r--r-- 1 dkp dkp 11 Nov 17 09:20 mu.txt -rw-r--r-- 1 dkp dkp 87137806 Nov 17 10:27 nodes_smooth.obj -rw-r--r-- 1 dkp dkp 67201344 Nov 17 00:15 parc.mif -rw-r--r-- 1 dkp dkp 50424384 Nov 17 00:15 parcRGB.mif -rw-r--r-- 1 dkp dkp 211 Nov 16 20:05 response_csf.txt -rw-r--r-- 1 dkp dkp 209 Nov 16 20:05 response_gm.txt -rw-r--r-- 1 dkp dkp 301 Nov 16 20:05 response_wm.txt lrwxrwxrwx 1 dkp dkp 38 Nov 16 20:28 rh.EC_average -> /opt/freesurfer/subjects/rh.EC_average -rw-r--r-- 1 dkp dkp 24 Nov 16 20:05 T1.json -rw-r--r-- 1 dkp dkp 1442468 Nov 16 20:05 T1_mask.mif -rw-r--r-- 1 dkp dkp 46138008 Nov 16 20:05 T1.mif -rw-r--r-- 1 dkp dkp 46137696 Nov 16 20:28 T1.nii -rw-r--r-- 1 dkp dkp 1948512 Nov 17 09:24 tdi_dwi.mif -rw-r--r-- 1 dkp dkp 748775556 Nov 17 09:51 tdi_hires.mif -rw-r--r-- 1 dkp dkp 46138176 Nov 17 09:31 tdi_T1.mif -rw-r--r-- 1 dkp dkp 29181510400 Nov 17 08:41 tractogram.tck -rw-r--r-- 1 dkp dkp 12685852 Nov 16 20:28 vis.mif -rw-r--r-- 1 dkp dkp 880835698 Nov 17 09:20 weights.csv

dkp@login2:mrtrix3_connectome.py-tmp-4MH5OG$

Let me know if I can provide anything else.

-Dianne

Dianne Patterson, Ph.D Speech, Language and Hearing Sciences, Room 314 dkp@arizona.edu

From: Robert Smith notifications@github.com Sent: Tuesday, November 17, 2020 2:44 PM To: BIDS-Apps/MRtrix3_connectome MRtrix3_connectome@noreply.github.com Cc: Patterson, Dianne K - (dkp) dkp@arizona.edu; Author author@noreply.github.com Subject: [EXT]Re: [BIDS-Apps/MRtrix3_connectome] output-verbosity 3 and tractogram (#91)

External Email

The tractogram file is certainly flagged to export at output verbosity 3https://github.com/BIDS-Apps/MRtrix3_connectome/blob/0.5.0/mrtrix3_connectome.py#L1947-L1950, and the conditions for exporthttps://github.com/BIDS-Apps/MRtrix3_connectome/blob/0.5.0/mrtrix3_connectome.py#L1946 are the same as those for the file you report seeinghttps://github.com/BIDS-Apps/MRtrix3_connectome/blob/0.5.0/mrtrix3_connectome.py#L1955-L1959; so I'm not sure exactly how it is that's happened. I'll try to reproduce locally... can you provide the exact conditions in which you're running (i.e. environment & command-line options)?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/BIDS-Apps/MRtrix3_connectome/issues/91#issuecomment-729232036, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAHLUQCUQIAE7GUUH6XWSZTSQLVCDANCNFSM4TZAORNA.

dkp commented 3 years ago

Actually, maybe I see the problem:

Your code is looking for: mrconvert: [ERROR] error opening image "tdi_t1.mif"

but the scratch directory contains tdi_T1.mif

-D

Dianne Patterson, Ph.D Speech, Language and Hearing Sciences, Room 314 dkp@arizona.edu

From: Patterson, Dianne K - (dkp) dkp@arizona.edu Sent: Tuesday, November 17, 2020 4:07 PM To: BIDS-Apps/MRtrix3_connectome reply@reply.github.com Subject: Re: [EXT]Re: [BIDS-Apps/MRtrix3_connectome] output-verbosity 3 and tractogram (#91)

Thanks Robert,

I'm attaching the job script I ran. The environment is our HPC with CentOS7.

The BIDS directory is called Nifti and contains a single subject (me) that I'm having the students process for the class.

ls Nifti CHANGES code dataset_description.json participants.tsv README sub-219 task-rest_bold.json

I just looked at the job log, and it seems to complain near the end (line 64).

And the tck file does appear to be in the scratch directory:

dkp@login2:mrtrix3_connectome.py-tmp-4MH5OG$ ls -l total 31263744

-rw-r--r-- 1 dkp dkp 49 Nov 16 20:28 5TT.json -rw-r--r-- 1 dkp dkp 63425740 Nov 16 20:28 5TT.mif -rw-r--r-- 1 dkp dkp 516800190 Nov 17 09:55 assignments.csv -rw-r--r-- 1 dkp dkp 1160607 Nov 17 09:54 connectome.csv -rw-r--r-- 1 dkp dkp 61545 Nov 16 20:05 dwi_mask.mif -rw-r--r-- 1 dkp dkp 62328376 Nov 16 20:05 dwi.mif -rw-r--r-- 1 dkp dkp 313 Nov 17 10:28 error.txt -rw-r--r-- 1 dkp dkp 68957516 Nov 17 10:27 exemplars.tck -rw-r--r-- 1 dkp dkp 128549844 Nov 16 20:19 FOD_WM.mif lrwxrwxrwx 1 dkp dkp 34 Nov 16 20:28 fsaverage -> /opt/freesurfer/subjects/fsaverage lrwxrwxrwx 1 dkp dkp 38 Nov 16 20:28 lh.EC_average -> /opt/freesurfer/subjects/lh.EC_average -rw-r--r-- 1 dkp dkp 4981 Nov 17 10:28 log.txt -rw-r--r-- 1 dkp dkp 1154830 Nov 17 09:58 meanlength.csv -rw-r--r-- 1 dkp dkp 11 Nov 17 09:20 mu.txt -rw-r--r-- 1 dkp dkp 87137806 Nov 17 10:27 nodes_smooth.obj -rw-r--r-- 1 dkp dkp 67201344 Nov 17 00:15 parc.mif -rw-r--r-- 1 dkp dkp 50424384 Nov 17 00:15 parcRGB.mif -rw-r--r-- 1 dkp dkp 211 Nov 16 20:05 response_csf.txt -rw-r--r-- 1 dkp dkp 209 Nov 16 20:05 response_gm.txt -rw-r--r-- 1 dkp dkp 301 Nov 16 20:05 response_wm.txt lrwxrwxrwx 1 dkp dkp 38 Nov 16 20:28 rh.EC_average -> /opt/freesurfer/subjects/rh.EC_average -rw-r--r-- 1 dkp dkp 24 Nov 16 20:05 T1.json -rw-r--r-- 1 dkp dkp 1442468 Nov 16 20:05 T1_mask.mif -rw-r--r-- 1 dkp dkp 46138008 Nov 16 20:05 T1.mif -rw-r--r-- 1 dkp dkp 46137696 Nov 16 20:28 T1.nii -rw-r--r-- 1 dkp dkp 1948512 Nov 17 09:24 tdi_dwi.mif -rw-r--r-- 1 dkp dkp 748775556 Nov 17 09:51 tdi_hires.mif -rw-r--r-- 1 dkp dkp 46138176 Nov 17 09:31 tdi_T1.mif -rw-r--r-- 1 dkp dkp 29181510400 Nov 17 08:41 tractogram.tck -rw-r--r-- 1 dkp dkp 12685852 Nov 16 20:28 vis.mif -rw-r--r-- 1 dkp dkp 880835698 Nov 17 09:20 weights.csv

dkp@login2:mrtrix3_connectome.py-tmp-4MH5OG$

Let me know if I can provide anything else.

-Dianne

Dianne Patterson, Ph.D Speech, Language and Hearing Sciences, Room 314 dkp@arizona.edu

From: Robert Smith notifications@github.com Sent: Tuesday, November 17, 2020 2:44 PM To: BIDS-Apps/MRtrix3_connectome MRtrix3_connectome@noreply.github.com Cc: Patterson, Dianne K - (dkp) dkp@arizona.edu; Author author@noreply.github.com Subject: [EXT]Re: [BIDS-Apps/MRtrix3_connectome] output-verbosity 3 and tractogram (#91)

External Email

The tractogram file is certainly flagged to export at output verbosity 3https://github.com/BIDS-Apps/MRtrix3_connectome/blob/0.5.0/mrtrix3_connectome.py#L1947-L1950, and the conditions for exporthttps://github.com/BIDS-Apps/MRtrix3_connectome/blob/0.5.0/mrtrix3_connectome.py#L1946 are the same as those for the file you report seeinghttps://github.com/BIDS-Apps/MRtrix3_connectome/blob/0.5.0/mrtrix3_connectome.py#L1955-L1959; so I'm not sure exactly how it is that's happened. I'll try to reproduce locally... can you provide the exact conditions in which you're running (i.e. environment & command-line options)?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/BIDS-Apps/MRtrix3_connectome/issues/91#issuecomment-729232036, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAHLUQCUQIAE7GUUH6XWSZTSQLVCDANCNFSM4TZAORNA.

dkp commented 3 years ago

Hi Robert, I tried rerunning the same pipeline without output verbosity 3 set. I left the existing mrtrix_scratch directory in place, hoping that its contents would be re-used to speed up rerunning the participant level. This does not seem to have happened though.

Unfortunately, it seems Freesurfer's recon-all has crashed under these circumstances. This is especially surprising because I used the same test dataset with fmriprep, and Freesurfer ran without issue. I will try wiping out the scratch directory and the participant directory and running again, then report back.

Thanks for all your hard work.

-Dianne mrtrix3_v0.5_participant_hcpmmp1.o212100.zip

Lestropie commented 3 years ago

I left the existing mrtrix_scratch directory in place, hoping that its contents would be re-used to speed up rerunning the participant level. This does not seem to have happened though.

When running outside of the container environment, MRtrix3 scripts have the -continue option, which is at least intended to provide such a functionality. However it's far from robust. Nominally such a functionality would be constructing a full graph of all processing steps, detecting which have been applied and which have not, and acting accordingly; whereas this literally just skips commands until a certain point in the script. But if you've not even utilised that option, then no, there's no way it'd re-use an existing scratch directory.

Unfortunately, it seems Freesurfer's recon-all has crashed under these circumstances. This is especially surprising because I used the same test dataset with fmriprep, and Freesurfer ran without issue.

It looks to me like an internal FreeSurfer failure:

mris_volmask --aseg_name aseg.presurf --label_left_white 2 --label_left_ribbon 3 --label_right_white 41 --label_right_ribbon 42 --save_ribbon --parallel freesurfer 
error: MatrixMultiply: m1 is null!

What I don't know is whether fmriprep may be reading such an error message and then making some modification that permits FreeSurfer to continue (I for instance do that myself with eddy; see here).

dkp commented 3 years ago

Thank you Robert,

I appreciate your time. I noticed that I had tarred up copies of the two MRtrix3 subdirectories in my derivatives folder. I've removed those now, and I'm trying the fresh run.

It is interesting to know about the -continue option. Eventually, I'll want to use the main MRtrix3 software, I'm sure....and that will be useful.

I have run the older MRtrix3_connectome 0.42 on this same dataset...and that appeared to run without any trouble. At least, it produced the following tree of files for me:

├── parc-hcpmmp1_lookup.txt └── sub-219 ├── anat │ ├── sub-219_5TT.nii.gz │ ├── sub-219_T1w.nii.gz │ ├── sub-219_brainmask.nii.gz │ ├── sub-219_parc-hcpmmp1.obj │ ├── sub-219_parc-hcpmmp1_colour.nii.gz │ ├── sub-219_parc-hcpmmp1_indices.nii.gz │ └── sub-219_tissues3D.nii.gz ├── connectome │ ├── sub-219_parc-hcpmmp1_assignments.csv │ ├── sub-219_parc-hcpmmp1_exemplars.tck │ ├── sub-219_parc-hcpmmp1_level-participant_connectome.csv │ └── sub-219_parc-hcpmmp1_meanlength.csv ├── dwi │ ├── eddyqc │ │ ├── eddy_movement_rms │ │ ├── eddy_outlier_map │ │ ├── eddy_outlier_n_sqr_stdev_map │ │ ├── eddy_outlier_n_stdev_map │ │ ├── eddy_outlier_report │ │ ├── eddy_parameters │ │ ├── eddy_post_eddy_shell_PE_translation_parameters │ │ ├── eddy_post_eddy_shell_alignment_parameters │ │ └── eddy_restricted_movement_rms │ ├── sub-219_brainmask.nii.gz │ ├── sub-219_bvalues.txt │ ├── sub-219_dwi.bval │ ├── sub-219_dwi.bvec │ ├── sub-219_dwi.nii.gz │ ├── sub-219_meanbzero.nii.gz │ ├── sub-219_model-tensor_fa.nii.gz │ ├── sub-219_tissue-WM_ODF.nii.gz │ └── sub-219_tissue-WM_response.txt └── tractogram ├── sub-219_mu.txt ├── sub-219_tractogram.tck ├── sub-219_variant-highres_tdi.nii.gz ├── sub-219_variant-native_tdi.nii.gz └── sub-219_weights.csv

I'll let you know if my tar file cleanup and fresh start of version 0.50 resolve the problem.

-Dianne

Dianne Patterson, Ph.D Speech, Language and Hearing Sciences, Room 314 dkp@arizona.edu

From: Robert Smith notifications@github.com Sent: Thursday, November 26, 2020 3:33 PM To: BIDS-Apps/MRtrix3_connectome MRtrix3_connectome@noreply.github.com Cc: Patterson, Dianne K - (dkp) dkp@arizona.edu; Author author@noreply.github.com Subject: [EXT]Re: [BIDS-Apps/MRtrix3_connectome] output-verbosity 3 and tractogram (#91)

External Email

I left the existing mrtrix_scratch directory in place, hoping that its contents would be re-used to speed up rerunning the participant level. This does not seem to have happened though.

When running outside of the container environment, MRtrix3 scripts have the -continue option, which is at least intended to provide such a functionality. However it's far from robust. Nominally such a functionality would be constructing a full graph of all processing steps, detecting which have been applied and which have not, and acting accordingly; whereas this literally just skips commands until a certain point in the script. But if you've not even utilised that option, then no, there's no way it'd re-use an existing scratch directory.

Unfortunately, it seems Freesurfer's recon-all has crashed under these circumstances. This is especially surprising because I used the same test dataset with fmriprep, and Freesurfer ran without issue.

It looks to me like an internal FreeSurfer failure:

mris_volmask --aseg_name aseg.presurf --label_left_white 2 --label_left_ribbon 3 --label_right_white 41 --label_right_ribbon 42 --save_ribbon --parallel freesurfer error: MatrixMultiply: m1 is null!

What I don't know is whether fmriprep may be reading such an error message and then making some modification that permits FreeSurfer to continue (I for instance do that myself with eddy; see herehttps://github.com/BIDS-Apps/MRtrix3_connectome/blob/0.5.0/mrtrix3_connectome.py#L1429-L1439).

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/BIDS-Apps/MRtrix3_connectome/issues/91#issuecomment-734497880, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAHLUQCTNFNQGJE2KWQVE53SR3JU5ANCNFSM4TZAORNA.

dkp commented 3 years ago

Sadly, even the cleaned up directory with version 0.5 crashes freesurfer ; ( -Dianne

Dianne Patterson, Ph.D Speech, Language and Hearing Sciences, Room 314 dkp@arizona.edu

From: Patterson, Dianne K - (dkp) dkp@arizona.edu Sent: Thursday, November 26, 2020 3:56 PM To: BIDS-Apps/MRtrix3_connectome reply@reply.github.com Subject: Re: [EXT]Re: [BIDS-Apps/MRtrix3_connectome] output-verbosity 3 and tractogram (#91)

Thank you Robert,

I appreciate your time. I noticed that I had tarred up copies of the two MRtrix3 subdirectories in my derivatives folder. I've removed those now, and I'm trying the fresh run.

It is interesting to know about the -continue option. Eventually, I'll want to use the main MRtrix3 software, I'm sure....and that will be useful.

I have run the older MRtrix3_connectome 0.42 on this same dataset...and that appeared to run without any trouble. At least, it produced the following tree of files for me:

├── parc-hcpmmp1_lookup.txt └── sub-219 ├── anat │ ├── sub-219_5TT.nii.gz │ ├── sub-219_T1w.nii.gz │ ├── sub-219_brainmask.nii.gz │ ├── sub-219_parc-hcpmmp1.obj │ ├── sub-219_parc-hcpmmp1_colour.nii.gz │ ├── sub-219_parc-hcpmmp1_indices.nii.gz │ └── sub-219_tissues3D.nii.gz ├── connectome │ ├── sub-219_parc-hcpmmp1_assignments.csv │ ├── sub-219_parc-hcpmmp1_exemplars.tck │ ├── sub-219_parc-hcpmmp1_level-participant_connectome.csv │ └── sub-219_parc-hcpmmp1_meanlength.csv ├── dwi │ ├── eddyqc │ │ ├── eddy_movement_rms │ │ ├── eddy_outlier_map │ │ ├── eddy_outlier_n_sqr_stdev_map │ │ ├── eddy_outlier_n_stdev_map │ │ ├── eddy_outlier_report │ │ ├── eddy_parameters │ │ ├── eddy_post_eddy_shell_PE_translation_parameters │ │ ├── eddy_post_eddy_shell_alignment_parameters │ │ └── eddy_restricted_movement_rms │ ├── sub-219_brainmask.nii.gz │ ├── sub-219_bvalues.txt │ ├── sub-219_dwi.bval │ ├── sub-219_dwi.bvec │ ├── sub-219_dwi.nii.gz │ ├── sub-219_meanbzero.nii.gz │ ├── sub-219_model-tensor_fa.nii.gz │ ├── sub-219_tissue-WM_ODF.nii.gz │ └── sub-219_tissue-WM_response.txt └── tractogram ├── sub-219_mu.txt ├── sub-219_tractogram.tck ├── sub-219_variant-highres_tdi.nii.gz ├── sub-219_variant-native_tdi.nii.gz └── sub-219_weights.csv

I'll let you know if my tar file cleanup and fresh start of version 0.50 resolve the problem.

-Dianne

Dianne Patterson, Ph.D Speech, Language and Hearing Sciences, Room 314 dkp@arizona.edu

From: Robert Smith notifications@github.com Sent: Thursday, November 26, 2020 3:33 PM To: BIDS-Apps/MRtrix3_connectome MRtrix3_connectome@noreply.github.com Cc: Patterson, Dianne K - (dkp) dkp@arizona.edu; Author author@noreply.github.com Subject: [EXT]Re: [BIDS-Apps/MRtrix3_connectome] output-verbosity 3 and tractogram (#91)

External Email

I left the existing mrtrix_scratch directory in place, hoping that its contents would be re-used to speed up rerunning the participant level. This does not seem to have happened though.

When running outside of the container environment, MRtrix3 scripts have the -continue option, which is at least intended to provide such a functionality. However it's far from robust. Nominally such a functionality would be constructing a full graph of all processing steps, detecting which have been applied and which have not, and acting accordingly; whereas this literally just skips commands until a certain point in the script. But if you've not even utilised that option, then no, there's no way it'd re-use an existing scratch directory.

Unfortunately, it seems Freesurfer's recon-all has crashed under these circumstances. This is especially surprising because I used the same test dataset with fmriprep, and Freesurfer ran without issue.

It looks to me like an internal FreeSurfer failure:

mris_volmask --aseg_name aseg.presurf --label_left_white 2 --label_left_ribbon 3 --label_right_white 41 --label_right_ribbon 42 --save_ribbon --parallel freesurfer error: MatrixMultiply: m1 is null!

What I don't know is whether fmriprep may be reading such an error message and then making some modification that permits FreeSurfer to continue (I for instance do that myself with eddy; see herehttps://github.com/BIDS-Apps/MRtrix3_connectome/blob/0.5.0/mrtrix3_connectome.py#L1429-L1439).

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/BIDS-Apps/MRtrix3_connectome/issues/91#issuecomment-734497880, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAHLUQCTNFNQGJE2KWQVE53SR3JU5ANCNFSM4TZAORNA.

Lestropie commented 3 years ago

Can you test running FreeSurfer directly on that image, i.e. without using fmriprep as a proxy? If fmriprep is detecting and resolving the issue, then a direct recon-all call should crash in the same way as it is when run within MRtrix3_connectome; if it doesn't crash in the same way, then there's something about how I'm configuring FreeSurfer within the container that it doesn't like.

dkp commented 3 years ago

Hi Robert,

I did not run recon-all by itself BUT, I did successfully run participant-level in version 0.5 on my dataset if I removed the session-related nesting from the dataset....no complaints from Freesurfer at all.

To be clear: This is fine: └── sub-219 ├── anat ├── dwi ├── fmap └── func

But this fails with freakouts from freesurfer: └── sub-219 └── ses-itbs ├── anat ├── dwi ├── fmap └── func

I tried none, hcpmmp1, and desikan. The results are consistent.

Thanks for your patience,

-Dianne

Dianne Patterson, Ph.D Speech, Language and Hearing Sciences, Room 314 dkp@arizona.edu

From: Robert Smith notifications@github.com Sent: Thursday, November 26, 2020 11:31 PM To: BIDS-Apps/MRtrix3_connectome MRtrix3_connectome@noreply.github.com Cc: Patterson, Dianne K - (dkp) dkp@arizona.edu; Author author@noreply.github.com Subject: [EXT]Re: [BIDS-Apps/MRtrix3_connectome] output-verbosity 3 and tractogram (#91)

External Email

Can you test running FreeSurfer directly on that image, i.e. without using fmriprep as a proxy? If fmriprep is detecting and resolving the issue, then a direct recon-all call should crash in the same way as it is when run within MRtrix3_connectome; if it doesn't crash in the same way, then there's something about how I'm configuring FreeSurfer within the container that it doesn't like.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/BIDS-Apps/MRtrix3_connectome/issues/91#issuecomment-734669583, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAHLUQAMX6JKFGHHBHQG3A3SR5BVBANCNFSM4TZAORNA.

Lestropie commented 3 years ago

I ran a session-nested dataset through participant-level processing with a FreeSurfer-based parcellation without issue, so it's not as simple as session-nesting borking FreeSurfer. It would be kinda weird for that to do so given that all FreeSurfer is provided with for recon-all is an already-renamed T1w image. I can only guess that in the process of converting from a session-nested BIDS structure to a more basic structure without sessions, you have somehow modified the T1w image in such a way that nullifies the original problem. Given I can't reproduce with data at hand I probably need access to the data you're using if I'm to diagnose.

dkp commented 3 years ago

Hi Robert,

This is strange. One other possible issue is that the session data is under Datalad control. I'll try running on data that has sessions but is not under datalad control and I'll let you know what happens.

In the mean time, here is access to the relevant datasets posted on OSF: sub_219_bids_datalad.zip https://osf.io/r5w93/ (this has a nested session) sub-219_bids_NoSess.zip https://osf.io/ps347/ (no sessions, no datalad)

When I "converted" I just moved the anat, dwi, fmap and func directories up a level and manually removed _ses-itbs from the names.

I'll let you know how my test of non-Datalad nested structure goes.

-Dianne

Dianne Patterson, Ph.D Speech, Language and Hearing Sciences, Room 314 dkp@arizona.edu

From: Robert Smith notifications@github.com Sent: Tuesday, December 1, 2020 4:05 PM To: BIDS-Apps/MRtrix3_connectome MRtrix3_connectome@noreply.github.com Cc: Patterson, Dianne K - (dkp) dkp@arizona.edu; Author author@noreply.github.com Subject: [EXT]Re: [BIDS-Apps/MRtrix3_connectome] output-verbosity 3 and tractogram (#91)

External Email

I ran a session-nested dataset through participant-level processing with a FreeSurfer-based parcellation without issue, so it's not as simple as session-nesting borking FreeSurfer. It would be kinda weird for that to do so given that all FreeSurfer is provided with for recon-all is an already-renamed T1w image. I can only guess that in the process of converting from a session-nested BIDS structure to a more basic structure without sessions, you have somehow modified the T1w image in such a way that nullifies the original problem. Given I can't reproduce with data at hand I probably need access to the data you're using if I'm to diagnose.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/BIDS-Apps/MRtrix3_connectome/issues/91#issuecomment-736875964, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAHLUQEQJOPSM2L2I3I7KQDSSVZELANCNFSM4TZAORNA.

dkp commented 3 years ago

It looks like my test dataset ran: It has nested sessions but is not under Datalad control. I've attached the successful job log.

Oddly, fmriprep manages to run freesurfer on the nested dataset even when it is under Datalad control. I'm happy to run other tests. I don't know how to interpret what I'm seeing here.

-Dianne

Dianne Patterson, Ph.D Speech, Language and Hearing Sciences, Room 314 dkp@arizona.edu

From: Robert Smith notifications@github.com Sent: Tuesday, December 1, 2020 4:05 PM To: BIDS-Apps/MRtrix3_connectome MRtrix3_connectome@noreply.github.com Cc: Patterson, Dianne K - (dkp) dkp@arizona.edu; Author author@noreply.github.com Subject: [EXT]Re: [BIDS-Apps/MRtrix3_connectome] output-verbosity 3 and tractogram (#91)

External Email

I ran a session-nested dataset through participant-level processing with a FreeSurfer-based parcellation without issue, so it's not as simple as session-nesting borking FreeSurfer. It would be kinda weird for that to do so given that all FreeSurfer is provided with for recon-all is an already-renamed T1w image. I can only guess that in the process of converting from a session-nested BIDS structure to a more basic structure without sessions, you have somehow modified the T1w image in such a way that nullifies the original problem. Given I can't reproduce with data at hand I probably need access to the data you're using if I'm to diagnose.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/BIDS-Apps/MRtrix3_connectome/issues/91#issuecomment-736875964, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAHLUQEQJOPSM2L2I3I7KQDSSVZELANCNFSM4TZAORNA.

Lestropie commented 3 years ago

Unfortunately without being able to reproduce the issue locally I'm quite stuck as to what to suggest. It's entirely possible that there is some stochastic behaviour within FreeSurfer that results in an outright error in one execution and no issue if the same data are re-run, which would make diagnosing the causative factor at your end even harder if that is the case. So I might just proceed with merging and tagging 0.5.1, and we can keep our eyes out for this issue (which should probably be moved to a separate issue since it's deviated from the original issue submission title).