Method for running subjects with many sessions (ie myconnectome)

[jbwexler]

@effigies Do you have a good understanding of how this will work? I think it was something like:

• First run entire subject with --anat-only. These should be stored somewhere in sourcedata or something?
• Then run in groups of several sessions and with --anat-derivatives flag (though I don't actually see how to break it down by session anywhere https://fmriprep.org/en/stable/usage.html)
• We will need to give specific instructions on how to utilize these datasets

[poldrack]

we would also like to run this using an average field map, since many sessions don't have field maps

On Fri, May 20, 2022 at 11:50 AM jbwexler @.***> wrote:

@effigies https://github.com/effigies Do you have a good understanding of how this will work? I think it was something like:

• First run entire subject with --anat-only. These should be stored somewhere in sourcedata or something?
• Then run in groups of several sessions and with --anat-derivatives flag (though I don't actually see how to break it down by session anywhere https://fmriprep.org/en/stable/usage.html)
• We will need to give specific instructions on how to utilize these datasets

— Reply to this email directly, view it on GitHub https://github.com/poldracklab/tacc-openneuro/issues/18, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGUVEC5SEP56AIEF7BX2STVK7NIBANCNFSM5WQJDNWA . You are receiving this because you are subscribed to this thread.Message ID: @.***>

-- Russell A. Poldrack Albert Ray Lang Professor of Psychology Associate Director, Stanford Data Science Director, SDS Center for Open and Reproducible Science Building 420 Stanford University Stanford, CA 94305

@. @.> http://www.poldracklab.org/

[effigies]

@poldrack @oesteban How have we done that in the past? I assume with some custom patches on the dataset itself...

[oesteban]

• First run entire subject with --anat-only. These should be stored somewhere in sourcedata or something?

This seems reasonable. I defer to @effigies for the YODA structuring of the outputs.

• Then run in groups of several sessions and with --anat-derivatives flag (though I don't actually see how to break it down by session anywhere https://fmriprep.org/en/stable/usage.html)

You would always set the same path for --anat-derivatives, depending on the previous response. Then launch a job-array (or launcher job list) where each task has a different bids-filter file, selecting one of the sessions.

we would also like to run this using an average field map, since many sessions don't have field maps

true, for this we would be missing a --fmap-derivatives flag, so that we can generate the average field map first and then the session-based functional processing will pick that one up.

If doing all sessions at once, this heuristic would be relatively easy to implement, but the session selection limits the fieldmaps the same way it would limit the anatomical workflow.

[oesteban]

How have we done that in the past? I assume with some custom patches on the dataset itself...

I've managed to do this:

1) Run --anat-only 2) Store the work directory in some persistent place 3) Submit jobs with the maximum wallclock time, and resubmit when the time's up, until it reached the end. Make sure each job generate separate log files. 4) Run a script to mine errors and other information from log files to understand what is missing.

[jbwexler]

I was talking to @effigies about this. He thinks these many-session datasets will take a lot of work to get right so we are considering putting this off for a while. I think we may run just --anat-only on them once fmriprep incorporates freesurfer 7 but put the rest off until we have more time to focus on them. If anyone disagrees or really needs fmriprep to be run any of the following datasets, let me know:

ds002685 ds001506 ds001246 ds001555 ds002674 ds000031 ds001499

[poldrack]

sounds fine to me

On Thu, May 26, 2022 at 7:13 AM jbwexler @.***> wrote:

I was talking to @effigies https://github.com/effigies about this. He thinks these many-session datasets will take a lot of work to get right so we are considering putting this off for a while. I think we may run just --anat-only on them once fmriprep incorporates freesurfer 7 but put the rest off until we have more time to focus on them. If anyone disagrees or really needs fmriprep to be run any of the following datasets, let me know:

ds002685 ds001506 ds003430 ds001246 ds001555 ds002674 ds000031 ds001499

— Reply to this email directly, view it on GitHub https://github.com/poldracklab/tacc-openneuro/issues/18#issuecomment-1138623340, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGUVEE2WZ6NMS7GOUTEW33VL6BKNANCNFSM5WQJDNWA . You are receiving this because you were mentioned.Message ID: @.***>

-- Russell A. Poldrack Albert Ray Lang Professor of Psychology Associate Director, Stanford Data Science Director, SDS Center for Open and Reproducible Science Building 420 Stanford University Stanford, CA 94305

@. @.> http://www.poldracklab.org/