Add containers for ABCD apps

yarikoptic commented 2 years ago

the ones behind https://brainlife.io/apps to go under https://github.com/ReproNim/containers/tree/master/images/abcd

figure out where API endpoint for listing them is (@soichih?)
while adding containers add metadata annotation for them (may be at git-annex level for now). I guess should be described somewhere within https://github.com/brainlife/abcd-spec

yarikoptic commented 2 years ago

Some notes from a chat with @soichih

The "entry point" for the ABCD app is not a "container" but rather the app itself which underneath might use container or multiple
the same container could be reused across multiple apps
brainlife CLI could be used to get information about apps (npm install brainlife ; bl login ; bl app query --json; didn't try -- need to recall login)
There is https://brainlife.io/api/warehouse/app/query endpoint .

We get 495 apps ATM

```shell ❯ curl --silent https://brainlife.io/api/warehouse/app/query | jq '.[].name' | nl | tail 486 "Compute summary statistics of diffusion measures mapped to cortical surface - Freesurfer stats - Deprecated Surface" 487 "app-anatomy" 488 "app-time-frequency" 489 "app-maxwell-filtering" 490 "app-filter" 491 "FSL Top-up & Eddy Correct" 492 "Extract individual tractograms for each segmented white matter tract" 493 "Freesurfer 7.3.2" 494 "Compute Backbone Network" 495 "app-events" ```

app record points in "github" to github user/repo.

here is selection of freesurfer apps with github repos for them

```shell ❯ cat /tmp/apps.json | jq '.[]| [.name, .github ] | @sh' | grep -i freesurfer | nl 1 "'Freesurfer' 'brainlife/app-freesurfer'" 2 "'Freesurfer on OSG (fsurf)' 'brainlife/app-freesurfer-osg'" 3 "'Freesurfer Deface' 'brainlife/app-deface'" 4 "'Freesurfer Labels to 3D Surfaces' 'kitchell/app-3Dfreesurfs'" 5 "'HCP Pipeline (PreFreeSurfer)' 'brainlife/app-hcp-prefreesurfer'" 6 "'HCP Pipeline (FreeSurfer)' 'brainlife/app-hcp-freesurfer'" 7 "'HCP Pipeline (PostFreeSurfer)' 'soichih/app-hcp-postfreesurfer'" 8 "'Convert Pial (Cortical) and White Matter Surfaces from Freesurfer' 'kitchell/app-3DPialWM_freesurfer'" 9 "'Freesurfer Group Analysis' 'brainlife/app-freesurfer-agg'" 10 "'Freesurfer on OSG' 'brainlife/app-freesurfer-osg'" 11 "'Freesurfer Statistics' 'brainlife/app-freesurfer-stats'" 12 "'Make white matter mask in dwi space from Freesurfer' 'brainlife/app-make-white-matter-mask-freesurfer'" 13 "'Make white matter mask in dwi space from Freesurfer (dtiinit)' 'brainlife/app-make-white-matter-mask-freesurfer'" 14 "'Parcellation Statistics - Deprecated' 'brainlife/app-freesurfer-stats'" 15 "'Compute summary statistics of diffusion measures from subcortical segmentation' 'brainlife/app-freesurfer-stats'" 16 "'Temp extract warp freesurferpost HCP' 'brainlife/app-extract-warp-temp'" 17 "'Generate ROIs for Visual White Matter Analysis in Freesurfer Space (Benson pRF)' 'brainlife/app-roiGenerator'" 18 "'Generate Visual Regions of Interest Binned by Eccentricity Estimates (Benson Atlas) - Freesurfer Space' 'brainlife/app-roiGenerator'" 19 "'Parcellation Statistics' 'brainlife/app-freesurfer-stats'" 20 "'Freesurfer 7.1.1' 'brainlife/app-freesurfer'" 21 "'Parcellation Statistics - Volume' 'brainlife/app-freesurfer-stats'" 22 "'Parcellation Statistics - Surface - Deprecated Datatype' 'brainlife/app-freesurfer-stats'" 23 "'Parcellation Statistics - Surface' 'brainlife/app-freesurfer-stats'" 24 "'Compute summary statistics of diffusion measures from Parcellation (volume)' 'brainlife/app-freesurfer-stats'" 25 "'Compute summary statistics of diffusion measures from Freesurfer Parcellation (volume)' 'brainlife/app-freesurfer-stats'" 26 "'Freesurfer 7.1.1 Longitudinal (Step1/2)' 'brainlife/app-freesurfer-longitudinal'" 27 "'Freesurfer 7.1.1 Longitudinal (Step 2/2)' 'brainlife/app-freesurfer-longitudinal'" 28 "'Freesurfer 7.1.1 - Multiple T1w Inputs' 'bacaron/app-freesurfer'" 29 "'Freesurfer Longitudinal Statistics' 'brainlife/app-freesurfer-longitudinal-statistics'" 30 "'Compute summary statistics of diffusion measures mapped to cortical surface - Freesurfer computations' 'brainlife/app-cortex-tissue-mapping-stats'" 31 "'Compute summary statistics of diffusion measures mapped to cortical surface - Freesurfer stats - Deprecated Surface' 'brainlife/app-cortex-tissue-mapping-stats'" 32 "'Freesurfer 7.3.2' 'brainlife/app-freesurfer'" ```

so the official seems to be the https://github.com/brainlife/app-freesurfer , and looking at https://github.com/brainlife/app-freesurfer/blob/master/main#L11 we see that execution happens via "straight" singularity exec ... docker://brainlife/freesurfer:7.3.2 ...

there is brainlife/ on standard docker hub, e.g. https://hub.docker.com/r/brainlife/freesurfer so in principle we could have archived all those images (https://github.com/ReproNim/containers/issues/48) and/or convert them to singularity but unless we come up with non-ad-hoc way to make use our singularity images instead of keeping converting from docker - benefit is unclear

With all that in mind, we might want

a dedicated "subdataset" for abcd apps which would then contain individual apps as subdatasets as well.
within in an archive of docker images from various locations accompanied with converted singularity images
... yet to analyze/figure out how we could "inject" ourselfves into main of the apps if at all

yarikoptic commented 11 months ago

just confirming on a sample one that no name of the image is stored anywhere in the metadata

❯ pwd
/tmp/app-dtiinit
❯ git grep neurodebian
main:singularity exec docker://brainlife/mcr:neurodebian1604-r2017a ./compiled/dtiinit
❯ curl --silent https://brainlife.io/api/warehouse/app/query | jq . | grep neurodebian1604-r2017a

so we would indeed need to have archive of all images. Ideally in OCI form from which we could then produce also singularity images or use directly with podman/docker

ReproNim / containers

Add containers for ABCD apps #74