Issue binding design file in singularity container

[bdeck8317]

Hi @a3sha2

Describe the bug Attempting to bind my data directory

Cohort file

id0,img
sub-10159,/home/bdeck8317_gmail_com/10159xcp_dat/fmriprep_dir/sub-10159/func/sub-10159_task-rest_space-T1w_desc-preproc_bold.nii.gz
...,...,...

Design File Paste your entire design (.dsn) file between the triple backticks

#!/usr/bin/env bash

###################################################################
#  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  #
###################################################################

###################################################################
# This design file stores the values of all variables required to
# execute a complete neuroimage processing pipeline. You may
# execute the analysis specified in this design file by calling
# (in any v4 or higher bash terminal):
#
# xcpEngine -d <design> -c <cohort> -o <output> <options>
#
# Variables fall into five general categories:
# * ANALYSIS VARIABLES are used at all stages of this analysis.
# * PIPELINE specifies the modules that comprise the analysis.
# * MODULE VARIABLES are used during one stage of the analysis.
#                  These are typically array variables with array
#                  indices equal to the index of the module that
#                  calls them.
# * OUTPUT VARIABLES may be used at all stages of the analysis.
#                  These are typically array variables with array
#                  indices equal to the value of the primary
#                  subject identifier. They will appear only in
#                  localised design files.
###################################################################

###################################################################
# ANALYSIS VARIABLES
###################################################################

analysis=fc_$(whoami)
design=/home/bdeck8317_gmail_com/$xcp_dir/fc-36p_despike.dsn
sequence=anatomical
standard=MNI%2x2x2

###################################################################
# PIPELINE
###################################################################

pipeline=prestats,confound2,regress,fcon,reho,alff,roiquant,norm,qcfc

###################################################################
# 1 PRESTATS
###################################################################

prestats_rerun[1]=1
prestats_cleanup[1]=1
prestats_process[1]=FMP

###################################################################
# 2 CONFOUND2
###################################################################

confound2_rps[2]=1
confound2_rms[2]=0
confound2_wm[2]=1
confound2_csf[2]=1
confound2_gsr[2]=1
confound2_acompcor[2]=0
confound2_tcompcor[2]=0
confound2_aroma[2]=0
confound2_past[2]=0
confound2_dx[2]=1
confound2_sq[2]=2
confound2_custom[2]=
confound2_censor[2]=0
confound2_censor_contig[2]=0
confound2_rerun[2]=1
confound2_cleanup[2]=1

###################################################################
# 3  REGRESS
###################################################################

regress_tmpf[3]=butterworth
regress_hipass[3]=0.01
regress_lopass[3]=0.08
regress_tmpf_order[3]=1
regress_tmpf_pass[3]=2
regress_tmpf_ripple[3]=0.5
regress_tmpf_ripple2[3]=20
regress_dmdt[3]=2
regress_1ddt[3]=1
regress_smo[3]=6
regress_sptf[3]=susan
regress_usan[3]=default
regress_usan_space[3]=
regress_rerun[3]=0
regress_cleanup[3]=1
regress_process[3]=DMT-DSP-TMP-REG

###################################################################
# 4 FCON
###################################################################

fcon_atlas[4]=all
fcon_metric[4]=corrcoef
fcon_rerun[4]=0
fcon_cleanup[4]=1

###################################################################
# 5 REHO
###################################################################

reho_nhood[5]=vertices
reho_roikw[5]=0 # does nothing at the moment
reho_sptf[5]=susan
reho_smo[5]=6
reho_rerun[5]=0
reho_cleanup[5]=1

###################################################################
# 6 ALFF
###################################################################

alff_hipass[6]=0.01
alff_lopass[6]=0.08
alff_sptf[6]=susan
alff_smo[6]=6
alff_rerun[6]=0
alff_cleanup[6]=1

###################################################################
# 7 ROIQUANT
###################################################################

roiquant_atlas[7]=all
roiquant_globals[7]=1
roiquant_vol[7]=0
roiquant_rerun[7]=0
roiquant_cleanup[7]=1

###################################################################
# 8 NORM
###################################################################
norm_primary[8]=1
norm_rerun[8]=0
norm_cleanup[8]=1

##################################################################
# 9 QCFC
###################################################################
qcfc_atlas[9]=power264
qcfc_sig[9]=fdr
qcfc_rerun[9]=0
qcfc_cleanup[9]=1

Error message Paste your error message between the backticks

Commandline: module load singularity
Running task 1
Commandline: singularity run -B /home/bdeck8317_gmail_com/10159xcp_dat xcpEngine.simg pennbbl/xcpengine:latest -t 3 -d /home/bdeck8317_gmail_com/10159xcp_dat/designs/fc-36p_despike.dsn -c /home/bdeck8317_gmail_com/10159xcp_dat/cohort/cohort.csv -o /home/bdeck8317_gmail_com/10159xcp_dat/xcp_out -i /home/bdeck8317_gmail_com/10159xcp_dat/tmpdir/
Received options: pennbbl/xcpengine:latest -t 3 -d /home/bdeck8317_gmail_com/10159xcp_dat/designs/fc-36p_despike.dsn -c /home/bdeck8317_gmail_com/10159xcp_dat/cohort/cohort.csv -o /home/bdeck8317_gmail_com/10159xcp_dat/xcp_out -i /home/bdeck8317_gmail_com/10159xcp_dat/tmpdir/
$XCPEDIR is /xcpEngine

Constructing a pipeline based on user specifications
····································································
· [D]Missing or empty design file
· [D]Aborting
___________________________________________________________________

   Usage: xcpEngine -d <design> <options>

Runtime Information Singularity

Additional context I know this is a binding issue. I have successfully run xcp via docker containers many times. I can't seem to determine what the issue is here and I would like to run using singularity to deploy batch jobs via slurm cluster.

For more context, here is my batch script:

# Parse the participants.tsv file and extract one subject ID from the line corresponding to this SLURM task.
#going to need to change bids dir to where the participants file is.
subject=$( sed -n -E "$((${SLURM_ARRAY_TASK_ID} + 1))s/sub-(\S*)\>.*/\1/gp" ./participants.tsv )

# make directories for subject specific bids files and fmriprep output
xcp_dir=$subject"xcp_dat"
fmriprep_dir=$xcp_dir/fmriprep_dir
tmpdir=$xcp_dir/tmpdir
xcp_out=$xcp_dir/xcp_out

mkdir –m777 $xcp_dir
mkdir –m777 $xcp_dir/fmriprep_dir
mkdir –m777 $xcp_dir/tmpdir
mkdir –m777 $xcp_dir/xcp_out
mkdir –m777 $xcp_dir/cohort
mkdir –m777 $xcp_dir/designs

# copy in fmriprep output

gsutil -m cp -r gs://fmriprep/*$subject* $xcp_dir/fmriprep_dir
gsutil -m cp -r gs://xcp_engine/*fc-36p_despike.dsn $xcp_dir/designs

# make design executable
chmod u+x $xcp_dir/designs/fc-36p_despike.dsn

# create individual cohort file
echo -e id0,img > $xcp_dir/cohort/cohort.csv

id0='sub-'$subject
img=$(find $HOME/$xcp_dir/fmriprep_dir/$id0/func -name "${id0}_task-rest_space-T1w_desc-preproc_bold.nii.gz")

echo -e $id0,$img >> $xcp_dir/cohort/cohort.csv

# Load singularity module for machine

loadmod="module load singularity"
echo Commandline: $loadmod
eval $loadmod

SINGULARITY_CMD="singularity run -B $HOME/$xcp_dir xcpEngine.simg "
# Compose the command line
cmd2="${SINGULARITY_CMD} pennbbl/xcpengine:latest -t 3 -d $HOME/$xcp_dir/designs/fc-36p_despike.dsn -c $HOME/$xcp_dir/cohort/cohort.csv -o $HOME/$xcp_dir/xcp_out -i $HOME/$xcp_dir/tmpdir/ "

# Setup done, run the command
echo Running task ${SLURM_ARRAY_TASK_ID}
echo Commandline: $cmd2
eval $cmd2

# Rsync to gcp bucket
gsutil -m rsync -r $xcp_dir/xcp_out gs://xcp_engine/

# Clean-up 
# rm -rf xcp_dir

exitcode=$?

# Output results to a table
echo "sub-$subject   ${SLURM_ARRAY_TASK_ID}    $exitcode" \
      >> ${SLURM_JOB_NAME}.${SLURM_ARRAY_JOB_ID}.tsv
echo Finished tasks ${SLURM_ARRAY_TASK_ID} with exit code $exitcode
exit $exitcode

Would appreciate any insights. Really do not understand what the issue is. Have also run singularity containers using batch scripts many times.

[pcamach2]

Hi @bdeck8317,

In your design file, the path to the design is design=/home/bdeck8317_gmail_com/$xcp_dir/fc-36p_despike.dsn

If I understand correctly and this script is running from your home directory, your batch script seems to save the designs to /home/bdeck8317_gmail_com/$xcp_dir/designs/fc-36p_despike.dsn

Is there a change in behavior if you change the design path in the dsn file to design=/home/bdeck8317_gmail_com/${xcp_dir}/designs/fc-36p_despike.dsn?

[bdeck8317]

unfortunately that doesn't solve the problem

[pcamach2]

@bdeck8317 I think I figured this one out. Maybe the error is being generated because the singularity run command for xcpEngine is trying to parse the docker image as an argument:

SINGULARITY_CMD="singularity run -B $HOME/$xcp_dir xcpEngine.simg "
# Compose the command line
cmd2="${SINGULARITY_CMD} pennbbl/xcpengine:latest -t 3 -d $HOME/$xcp_dir/designs/fc-36p_despike.dsn -c $HOME/$xcp_dir/cohort/cohort.csv -o $HOME/$xcp_dir/xcp_out -i $HOME/$xcp_dir/tmpdir/ "

should work when changed to:

SINGULARITY_CMD="singularity run -B $HOME/$xcp_dir xcpEngine.simg "
# Compose the command line
cmd2="${SINGULARITY_CMD} -t 3 -d $HOME/$xcp_dir/designs/fc-36p_despike.dsn -c $HOME/$xcp_dir/cohort/cohort.csv -o $HOME/$xcp_dir/xcp_out -i $HOME/$xcp_dir/tmpdir/ "

I hope this helps!

[bdeck8317]

@pcamach2 wow, yup, that did it. Thanks so much for pointing that out :/