I/O spec for AFNI BIDS outputs

tsalo commented 4 weeks ago

Does anyone know what AFNI BIDS derivatives look like?

handwerkerd commented 3 weeks ago

AFNI outputs a file name registry that can be mapped onto any file name structure. I know there was work to make a BIDS mapping, but I'm not sure of the current status. @mrneont might know more details.

mrneont commented 3 weeks ago

@tsalo : sorry for delay in reply, been away on leave.

The main FMRI pipeline tool in AFNI is afni_proc.py. This is typically used to fully process a single subject dataset through regression modeling (GLM). First, afni_proc.py generates a commented script that records all the processing that will take place, and running that script generates a standard results directory of both copied, intermediate and final data, as well as QC HTML and other quantitative records of the processing.

Since afni_proc.py can be used pretty flexibly (surface or volume; ROI-based or voxelwise; final space being template, or subject anatomical, or subject EPI; including blip up/down or not; etc.), there can be a wide array of outputs. To help keep track of all this, the afni_proc.py script generates a dictionary of meaningful outputs each time. This is called the "uvar dictionary", where the keys are variables for the user to know about what the output of interest is. This is described in this recent paper:

Taylor PA, Glen DR, Chen G, Cox RW, Hanayik T, Rorden C, Nielson DM, Rajendra JK, Reynolds RC (2024). A Set of FMRI Quality Control Tools in AFNI: Systematic, in-depth and interactive QC with afni_proc.py and more. doi: 10.1101/2024.03.27.586976.
https://direct.mit.edu/imag/article/doi/10.1162/imag_a_00246/123633/A-Set-of-FMRI-Quality-Control-Tools-in-AFNI ... particularly Table 2. The full list of possible items (from which a given run of afni_proc.py will have a subset) is also displayable via gen_ss_review_scripts.py -show_uvar_dict.

That being said, AFNI now also has a -bids_deriv option, whereby you can generate an additional output directory that contains a structure of output in the current BIDS Derivative for FMRI format. This is described in the latest addition of the afni_proc.py draft (might have to wait for the arXiv update which was submitted earlier this week to update) here:

Reynolds RC, Glen DR, Chen G, Saad ZS, Cox RW, Taylor PA (2024). Processing, evaluating and understanding FMRI data with afni_proc.py.
https://arxiv.org/abs/2406.05248

In recent discussion with BIDS developers at OHBM Brainhack, we discussed that if other FMRI pipeline tools had a dictionary of meaningful outputs, analogous to afni_proc.py's uvar dictionary, then it would be straightforward to map all outputs across pipelines. In some sense, we have already worked on a mapping from afni_proc.py to current BIDS Derivatives, because we can map the afni_proc.py uvars to a given naming structure that BIDS Derivatives has defined. Dictionaries would likely overlap partially and not completely, because different pipelines do different things (such is the case with afni_proc.py and current BIDS Derivatives), but that is certainly fine. Having a mapping between the major outputs seems the key thing, because these are adaptable even if things like filenames (which are the essentially values in the dictionary) change.

Happy to hear your thoughts on this.

--pt

tsalo commented 2 weeks ago

Thanks @mrneont. I think I misunderstood the --bids_deriv parameter. I thought that it would write out all of the afni_proc.py derivatives in a BIDS-like format, but it looks like --bids_deriv only writes out a subset of files (below is the relevant part of the tree from my test run earlier today):

    ├── bids_deriv
    │   └── sub-23638
    │       ├── anat
    │       │   ├── sub-23638-MNI_desc-preproc_T1w.nii.gz
    │       │   ├── sub-23638_space-anat_T1w.nii.gz
    │       │   └── tpl-MNI152NLin6Asym_res-02_desc-brain_T1w.nii.gz
    │       └── func
    │           ├── sub-23638_task-TASKNAME_space-MNI_desc-brain_mask.nii.gz
    │           ├── sub-23638_task-TASKNAME_space-MNI_desc-resid_bold.nii.gz
    │           ├── sub-23638_task-TASKNAME_space-MNI_stat-residtsnr_statmap.nii.gz
    │           ├── sub-23638_task-TASKNAME_space-orig_boldref.nii.gz
    │           └── sub-23638_task-TASKNAME_space-orig_desc-tcat_bold.nii.gz

I will take a look at the uvars dictionary to see if that's what I should reference in fMRIPost-tedana. I might write an ingression script to convert the outputs to a BIDS-like format based on that dictionary, so fMRIPost-tedana can just use pybids to collect the necessary files on its end.

mrneont commented 2 weeks ago

Hi, @tsalo : Adding "-bids_deriv yes" to the afni_proc.py command means that all the afni_proc.py outputs that exist in a BIDS Derivative naming convention are mapped to that. There are many other afni_proc.py outputs that just don't have a known home there at present---though please let us know if you think there are others that are ready to be mapped---and indeed I suspect there might be other BIDS Derivatives outputs from other software that afni_proc.py doesn't make. If there are some that might be addable, please also ping and we can discuss if they could be added when this option is used.

It should also be noted that the existing afni_proc.py-to-BIDS-Derivatives mapping was done with invaluable the help of Chris Marciewicz and Dylan Nielson.

Checking out the uvars sounds good---that is the set of major afni_proc.py outputs from processing, which are used in generating the APQC HTML report, as well as mapping to BIDS Derivatives. Happy to hear your thoughts on this.

ME-ICA / fmripost-tedana

I/O spec for AFNI BIDS outputs #6