config system and basic automated workflow setup

[vanandrew]

This pull request updates the omni dependency and adds a --skip_synth flag to disable Synth Distortion Correction, which only applies distortion correction from topup. In addition, it adds configuration file support and includes a basic continuous integration setup with circleci (though some parameters + config will be need on the compute end of things to get it truly working).

Config files

The config system uses the toml format. Arguments can be specified in the config file, as shown in the following example:

[nhp-abcd-bids-pipeline]
subject_list = [ "baloo",]
collect = false
ncpus = 1
freesurfer_license = "/data/license.txt"
bandstop = [ 20.0, 28.0,]
max_cortical_thickness = 5
norm_gm_std_dev_scale = 0.5
norm_wm_std_dev_scale = 0.5
norm_csf_std_dev_scale = 0.5
make_white_from_norm_t1 = true
single_pass_pial = false
t1_brain_mask = "/data/dcan_macaque_T1w_masks/sub-baloo_ses-20210513_T1w_pre_mask_edit.nii.gz"
t1_reg_method = "ANTS_NO_INTERMEDIATE"
check_outputs_only = false
print = false
ignore_expected_outputs = false
multi_template_dir = "/opt/pipeline/global/templates/JointLabelCouncil"
norm_method = "ROI_IPS"
skip_synth = false

which can be passed into the program as:

docker run -it --rm \
-v ${NHP_DATA}:/data \
-v ${NHP_OUTPUT}:/output \
dcanumn/nhp-abcd-bids-pipeline \
/data /output --config_file config.toml

this is functionally equivalent to:

docker run -it --rm \
-v ${NHP_DATA}:/data \
-v ${NHP_OUTPUT}:/output \
dcanumn/nhp-abcd-bids-pipeline \
/data /output \
--freesurfer-license /data/license.txt \
--participant-label baloo \
--t1-brain-mask /data/dcan_macaque_T1w_masks/sub-baloo_ses-20210513_T1w_pre_mask_edit.nii.gz \
--t1-reg-method ANTS_NO_INTERMEDIATE \
--hyper-normalization-method ROI_IPS \
--norm-gm-std-dev-scale 0.5 \
--norm-wm-std-dev-scale 0.5 \
--norm-csf-std-dev-scale 0.5 \
--make-white-from-norm-t1 \
--bandstop 20 28 \
--multi-template-dir /opt/pipeline/global/templates/JointLabelCouncil

Arguments on the command-line take precedence over any arguments defined in the file, for example:

docker run -it --rm \
-v ${NHP_DATA}:/data \
-v ${NHP_OUTPUT}:/output \
dcanumn/nhp-abcd-bids-pipeline \
/data /output --config_file config.toml \
--ncpus 8

Will use ncpus=8 rather than ncpus=1 as in the config file.

CI with circleci

This pull request contains a docker compose config for setting up a self-hosted runner for the circleci service. It can be run by copying over env file to the project root directory, and running docker compose up -d. This will register a dockerized runner with the circleci service (you will need to register a CIRCLECI_API_TOKEN for this to work).

The self-hosted runner passes the docker.sock socket from the host, so that docker images inside the runner can be used. When the workflow is triggered the pipeline docker image can use NHP_DATA_MOUNT and NHP_WRITABLE_MOUNT to read and write data to the host.

Host Machine ------------------------------------> circleci runner ------------------------------------> nhp-abcd-bids-pipeline Docker                                     Docker (mounting host docker.sock)