✨ Incorporate correlation matrix computation options

[gkiar]

Related problem

n/a

Proposed feature

I want to automatically generate FCC matrices at the end of my preprocessing. There are several ways to do this without adding dependencies within the CPAC environment and stack:

1. AFNI only (3DNetCorr)
2. Python only (Nilearn)
3. Hybrid (3dROIstats + Nilearn/C-PAC wrapper)

Acceptance criteria

Here are example implementations for each of those solutions:

1. AFNI only

3dNetCorr -inset <timeseries>  -in_rois <parcellation> -prefix <tmp_output>
tail -n +7 <tmp_output>_000.netcc > <output>  # This removes the header info

2. Python only

from argparse import ArgumentParser

from nilearn.input_data import NiftiLabelsMasker
from nilearn.connectome import ConnectivityMeasure
from nilearn import plotting
import numpy as np

parser = ArgumentParser()
parser.add_argument("parcellation")
parser.add_argument("timeseries")
parser.add_argument("output")

args = parser.parse_args()

masker = NiftiLabelsMasker(labels_img=args.parcellation, standardize=True, verbose=True)
timeser = masker.fit_transform(args.timeseries)
correlation_measure = ConnectivityMeasure(kind='correlation')
corr_matrix = correlation_measure.fit_transform([timeser])[0]
np.fill_diagonal(corr_matrix, 0)
np.savetxt(args.output, corr_matrix)

3. Hybrid

3dROIstats -quiet -mask <parcellation> -1Dformat <timeseries> > <tmp_output1>
1dtranspose <tmp_output1> > <tmp_output2>

python -c "import CPAC.connectome.pipeline as cc; cc.compute_correlation('<tmp_output2>}', 'PearsonCorr')"
python -c "import numpy as np; d=np.load('correlation_connectome.npy'); np.savetxt('<output>', d)"

rm correlation_connectome.npy  # Note: CPAC Currently imposes this 

Alternatives

No response

Additional context

No response

[shnizzedy]

Note to whoever tackles this feature: You'll probably want to add linux_openmp_64/3dNetCorr to dev/docker_data/required_afni_pkgs.txt to have 3dNetCorr available in the image

[sgiavasis]

Regarding implementing both 1 and 2, @gkiar - any suggestions on how to name them and present them as options in the pipe config?

Even this would work:

correlation_matrix:
  using: ['AFNI', 'Nilearn']

[gkiar]

We probably want connectivity_measure and connectivity_implementation.

For measure we support Pearson, Spearman, and maybe MGC (all eventually, just Pearson for now is great).

For implementation, exactly as you said, AFNI or Nilearn

[sgiavasis]

Okay, so I'm picturing:

connectivity_matrices:
  using: ['AFNI, 'Nilearn']
  measure: ['Pearson', 'Spearman', 'MGC']     # (eventually)

And for now, since only Pearson:

connectivity_matrices:
  using: ['AFNI, 'Nilearn']

[gkiar]

Love it. Do we want matrix or matrices? Maybe even just connectivity to avoid ambiguity surrounding plurality?

[shnizzedy]

Are these matrices exclusively from timeseries outputs, or also from SCA outputs?

[shnizzedy]

Love it. Do we want matrix or matrices? Maybe even just connectivity to avoid ambiguity surrounding plurality?

[sgiavasis]

Are these matrices exclusively from timeseries outputs, or also from SCA outputs?

Timeseries outputs, as those are the "raw" (but preprocessed) time series for each parcel.

[shnizzedy]

In #1595 right now we're setting https://github.com/FCP-INDI/C-PAC/blob/f9e6bfaefbd295e5d007969c69e06b1ffe83d83c/dev/docker_data/default_pipeline.yml#L1488-L1496 like https://github.com/FCP-INDI/C-PAC/issues/1559#issuecomment-947085618, but would

timeseries_extraction:

  # Options:
  #  ['AFNI', 'Nilearn']
  using:

connectivity_matrix:
  # Create a connectivity matrix from timeseries data

  # Options:
  #  ['Pearson', 'Partial', 'Spearman']
  measure:

make more sense? Then we'd output separate *.timeseries.1D files for AFNI and Nilearn if both are selected and make the connectivity matrices based on those