Does not complete

[clarkenj]

I have tried re-installing with a fresh environment. I have pasted below what I did for the install and then running it. Please let me know what I'm doing wrong!

Install:

cd /lustre03/project/6003287/nclarke/.virtualenvs
module load python
python -m venv giga_auto_qc
source giga_auto_qc/bin/activate
cd ~/projects/rrg-pbellec/nclarke/giga_preprocess2/
git clone git@github.com:SIMEXP/giga_auto_qc.git
cd giga_auto_qc/
git tag
git checkout 0.3.1
pip install -e .

I ran giga_auto_qc --help and that showed the parameters.

I then ran the following script (which is in ~/projects/rrg-pbellec/nclarke/giga_preprocess2)

#!/bin/bash
#SBATCH --account=def-pbellec
#SBATCH --job-name=cimaq_qc
#SBATCH --output=/lustre04/scratch/nclarke/logs/cimaq_qc.%a.out
#SBATCH --error=/lustre04/scratch/nclarke/logs/cimaq_qc.%a.err
#SBATCH --time=02:00:00
#SBATCH --cpus-per-task=1
#SBATCH --mem=2G

source /lustre03/project/6003287/nclarke/.virtualenvs/giga_auto_qc/bin/activate

FMRIPREP_PATH=/home/nclarke/scratch/cimaq_fmriprep-20.2.7lts_1687549726/CIMAQ
QC_OUTPUT=/lustre04/scratch/${USER}/cimaq_qc

mkdir -p $QC_OUTPUT

echo ${FMRIPREP_PATH}
if [ -d "${FMRIPREP_PATH}/fmriprep-20.2.7lts" ]; then
    mkdir -p ${QC_OUTPUT}
    rm ${FMRIPREP_PATH}/fmriprep-20.2.7lts/layout_index.sqlite
    giga_auto_qc \
        ${FMRIPREP_PATH}/fmriprep-20.2.7lts \
        ${QC_OUTPUT} \
        participant

else
    echo "no preprocessed data for ${FMRIPREP_PATH}"
fi

The output is just echos the fmriprep path (/home/nclarke/scratch/cimaq_fmriprep-20.2.7lts_1687549726/CIMAQ), and times out.

I also tried doing:

cd ~/projects/rrg-pbellec/nclarke/giga_preprocess2/
salloc --account=rrg-pbellec --time=02:00:00
source /lustre03/project/6003287/nclarke/.virtualenvs/giga_auto_qc/bin/activate
giga_auto_qc /home/nclarke/scratch/cimaq_fmriprep-20.2.7lts_1687549726/CIMAQ/fmriprep-20.2.7lts /home/nclarke/scratch/cimaq_qc participant --reindex-bids

Output (and then times out):

Namespace(bids_dir=PosixPath('/home/nclarke/scratch/cimaq_fmriprep-20.2.7lts_1687549726/CIMAQ/fmriprep-20.2.7lts'), output_dir=PosixPath('/home/nclarke/scratch/cimaq_qc'), analysis_level='participant', participant_label=None, session=None, task=None, quality_control_parameters=None, reindex_bids=True, verbose=1)
Quality control parameters: {'mean_fd': 0.55, 'scrubbing_fd': 0.2, 'proportion_kept': 0.5, 'anatomical_dice': 0.97, 'functional_dice': 0.89}

[htwangtw]

If the problem is time out, increase time given to the job

[htwangtw]

I will close this issue as this has nothing to do with the BIDS-app itself but a slurm usage question.

[clarkenj]

If the problem is time out, increase time given to the job

When I asked before you said that if it does not complete in under an hour there is something wrong, as ABIDE does and there are multiple sites, so I gave it two hours. I'll increase further.

[htwangtw]

Should have clarified - I was running qc per site, and for the bigger sites it can take longer. Always check the number of the subjects and scale accordingly, there's no good rule of thumbs here unfortunately.