search

neurogenomics / CUT_n_TAG

Preprocessing pipeline for CUT&TAG data.
MIT License
0 stars 0 forks source link

nf-core/atacseq on HPC: "Job resource selection does not match any permitted configuration" #5

Closed bschilder closed 3 years ago

bschilder commented 3 years ago

It looks I'm still having some issues submitting jobs with my current nf config file. Running nf-core/atacseq on Di's CUT&TAG data now gets past the initial steps (checking my design file) but fails after that. @Combiz, can you see anything here that I'm doing wrong? i should have access to medbio now, but to simplify things for the moment (and to make sure others can use my script) I don't specify any queues. Is nf able to select the appropriate queue or do i have to tell it which one?

hpc nextflow config file

//Profile config names for nf-core/configs

params {
  // Config Params
  config_profile_description = 'Imperial College London - DRI - HPC Profile -- provided by nf-core/configs.'
  config_profile_contact = 'Combiz Khozoie (c.khozoie@imperial.ac.uk)'
  config_profile_url = 'http://www.github.com/combiz/scFlow'

  // Resources
  max_memory = 640.GB
  max_cpus = 32
  max_time = 168.h

  // Analysis Resource Params
  ctd_folder = "/rds/general/user/$USER/projects/ukdrmultiomicsproject/live/Analyses/scFlowResources/refs/ctd"
  ensembl_mappings = "/rds/general/user/$USER/projects/ukdrmultiomicsproject/live/Analyses/scFlowResources/src/ensembl-ids/ensembl_mappings.tsv"

}

executor {
    $pbspro {
        queueSize = 50
    }

    $local {
          cpus = 2
        queueSize = 1
        memory = '32 GB'
    }
}

singularity {
  enabled = true
  autoMounts = true
  cacheDir = "/rds/general/user/$USER/projects/neurogenomics-lab/live/.singularity-cache"
  runOptions = "-B /rds/,/rdsgpfs/,/rds/general/user/$USER/ephemeral/tmp/:/tmp,/rds/general/user/$USER/ephemeral/tmp/:/var/tmp"

}

process {
  // NOTE: queue lines are commented out bc you first need to request access to these special queues from RCS.
  beforeScript = 'module load singularity'
  executor = 'pbspro'
  // queue = 'pqmedbio-tput'

  //queue = 'med-bio' !! this is an alias and shouldn't be used

  //withLabel:process_large {
  //  queue = 'pqmedbio-large'
  // }

}

workDir = "/rds/general/user/$USER/ephemeral/tmp"

nf-core/atacseq output

-bash-4.2$ bash run_atacseq.sh 
N E X T F L O W  ~  version 20.10.0
Launching `nf-core/atacseq` [compassionate_saha] - revision: 1b3a832db5 [1.2.1]
----------------------------------------------------
                                        ,--./,-.
        ___     __   __   __   ___     /,-._.--~'
  |\ | |__  __ /  ` /  \ |__) |__         }  {
  | \| |       \__, \__/ |  \ |___     \`-._,-`-,
                                        `._,._,'
  nf-core/atacseq v1.2.1
----------------------------------------------------
Run Name             : compassionate_saha
Data Type            : Paired-End
Design File          : /rds/general/user/bms20/home/neurogenomics/GitRepos/CUT_n_TAG/raw_data/HK5M2BBXY/design_noindex2.csv
Genome               : GRCh37
Fasta File           : s3://ngi-igenomes/igenomes//Homo_sapiens/Ensembl/GRCh37/Sequence/WholeGenomeFasta/genome.fa
GTF File             : s3://ngi-igenomes/igenomes//Homo_sapiens/Ensembl/GRCh37/Annotation/Genes/genes.gtf
Gene BED File        : s3://ngi-igenomes/igenomes//Homo_sapiens/Ensembl/GRCh37/Annotation/Genes/genes.bed
BWA Index            : s3://ngi-igenomes/igenomes//Homo_sapiens/Ensembl/GRCh37/Sequence/BWAIndex/genome.fa
Blacklist BED        : /rds/general/user/bms20/home/.nextflow/assets/nf-core/atacseq/assets/blacklists/GRCh37-blacklist.bed
Mitochondrial Contig : MT
MACS2 Genome Size    : 2.7e9
Min Consensus Reps   : 1
MACS2 Narrow Peaks   : Yes
Trim R1              : 0 bp
Trim R2              : 0 bp
Trim 3' R1           : 0 bp
Trim 3' R2           : 0 bp
NextSeq Trim         : 0 bp
Fingerprint Bins     : 500000
Save Genome Index    : No
Max Resources        : 640 GB memory, 32 cpus, 7d time per job
Container            : singularity - /rds/general/user/bms20/home/atacseq_latest.sif
Output Dir           : /rds/general/user/bms20/home/neurogenomics/GitRepos/CUT_n_TAG/processed_data/HK5M2BBXY
Launch Dir           : /rds/general/project/neurogenomics-lab/live/GitRepos/CUT_n_TAG
Working Dir          : /rds/general/user/bms20/ephemeral/tmp
Script Dir           : /rds/general/user/bms20/home/.nextflow/assets/nf-core/atacseq
User                 : bms20
Config Profile       : standard
Config Description   : Imperial College London - DRI - HPC Profile -- provided by nf-core/configs.
Config Contact       : Combiz Khozoie (c.khozoie@imperial.ac.uk)
Config URL           : http://www.github.com/combiz/scFlow
----------------------------------------------------
executor >  pbspro (3)
[df/f52120] process > CHECK_DESIGN (design_noindex2.csv) [100%] 1 of 1 ✔
[-        ] process > MAKE_TSS_BED                       -
[-        ] process > MAKE_GENOME_FILTER                 -
[67/8de9ff] process > FASTQC (control_R2_T1)             [  0%] 0 of 2
[-        ] process > TRIMGALORE                         [  0%] 0 of 2
[-        ] process > BWA_MEM                            -
[-        ] process > SORT_BAM                           -
[-        ] process > MERGED_LIB_BAM                     -
[-        ] process > MERGED_LIB_BAM_FILTER              -
[-        ] process > MERGED_LIB_BAM_REMOVE_ORPHAN       -
[-        ] process > MERGED_LIB_PRESEQ                  -
[-        ] process > MERGED_LIB_PICARD_METRICS          -
[-        ] process > MERGED_LIB_BIGWIG                  -
[-        ] process > MERGED_LIB_PLOTPROFILE             -
[-        ] process > MERGED_LIB_PLOTFINGERPRINT         -
[-        ] process > MERGED_LIB_MACS2                   -
[-        ] process > MERGED_LIB_MACS2_ANNOTATE          -
[-        ] process > MERGED_LIB_MACS2_QC                -
executor >  pbspro (3)
[df/f52120] process > CHECK_DESIGN (design_noindex2.csv) [100%] 1 of 1 ✔
[-        ] process > MAKE_TSS_BED                       -
[-        ] process > MAKE_GENOME_FILTER                 -
[67/8de9ff] process > FASTQC (control_R2_T1)             [  0%] 0 of 2
[30/61b27f] process > TRIMGALORE (control_R2_T1)         [ 50%] 1 of 2, failed: 1
[-        ] process > BWA_MEM                            -
[-        ] process > SORT_BAM                           -
[-        ] process > MERGED_LIB_BAM                     -
[-        ] process > MERGED_LIB_BAM_FILTER              -
[-        ] process > MERGED_LIB_BAM_REMOVE_ORPHAN       -
[-        ] process > MERGED_LIB_PRESEQ                  -
[-        ] process > MERGED_LIB_PICARD_METRICS          -
[-        ] process > MERGED_LIB_BIGWIG                  -
[-        ] process > MERGED_LIB_PLOTPROFILE             -
[-        ] process > MERGED_LIB_PLOTFINGERPRINT         -
[-        ] process > MERGED_LIB_MACS2                   -
[-        ] process > MERGED_LIB_MACS2_ANNOTATE          -
[-        ] process > MERGED_LIB_MACS2_QC                -
[-        ] process > MERGED_LIB_CONSENSUS               -
executor >  pbspro (3)
[df/f52120] process > CHECK_DESIGN (design_noindex2.csv) [100%] 1 of 1 ✔
[-        ] process > MAKE_TSS_BED                       [  0%] 0 of 1
[-        ] process > MAKE_GENOME_FILTER                 [  0%] 0 of 1
[67/8de9ff] process > FASTQC (control_R2_T1)             [  0%] 0 of 2
[30/61b27f] process > TRIMGALORE (control_R2_T1)         [ 50%] 1 of 2, failed: 1
[-        ] process > BWA_MEM                            -
[-        ] process > SORT_BAM                           -
[-        ] process > MERGED_LIB_BAM                     -
[-        ] process > MERGED_LIB_BAM_FILTER              -
[-        ] process > MERGED_LIB_BAM_REMOVE_ORPHAN       -
[-        ] process > MERGED_LIB_PRESEQ                  -
[-        ] process > MERGED_LIB_PICARD_METRICS          -
[-        ] process > MERGED_LIB_BIGWIG                  -
[-        ] process > MERGED_LIB_PLOTPROFILE             -
[-        ] process > MERGED_LIB_PLOTFINGERPRINT         -
[-        ] process > MERGED_LIB_MACS2                   -
[-        ] process > MERGED_LIB_MACS2_ANNOTATE          -
[-        ] process > MERGED_LIB_MACS2_QC                -
[-        ] process > MERGED_LIB_CONSENSUS               -
executor >  pbspro (3)
[df/f52120] process > CHECK_DESIGN (design_noindex2.csv) [100%] 1 of 1 ✔
[-        ] process > MAKE_TSS_BED                       [  0%] 0 of 1
[-        ] process > MAKE_GENOME_FILTER                 [  0%] 0 of 1
[67/8de9ff] process > FASTQC (control_R2_T1)             [  0%] 0 of 2
[30/61b27f] process > TRIMGALORE (control_R2_T1)         [ 50%] 1 of 2, failed: 1
[-        ] process > BWA_MEM                            -
[-        ] process > SORT_BAM                           -
[-        ] process > MERGED_LIB_BAM                     -
[-        ] process > MERGED_LIB_BAM_FILTER              -
[-        ] process > MERGED_LIB_BAM_REMOVE_ORPHAN       -
[-        ] process > MERGED_LIB_PRESEQ                  -
[-        ] process > MERGED_LIB_PICARD_METRICS          -
[-        ] process > MERGED_LIB_BIGWIG                  -
[-        ] process > MERGED_LIB_PLOTPROFILE             -
[-        ] process > MERGED_LIB_PLOTFINGERPRINT         -
[-        ] process > MERGED_LIB_MACS2                   -
[-        ] process > MERGED_LIB_MACS2_ANNOTATE          -
[-        ] process > MERGED_LIB_MACS2_QC                -
[-        ] process > MERGED_LIB_CONSENSUS               -
[-        ] process > MERGED_LIB_CONSENSUS_ANNOTATE      -
executor >  pbspro (3)
[df/f52120] process > CHECK_DESIGN (design_noindex2.csv) [100%] 1 of 1 ✔
[-        ] process > MAKE_TSS_BED                       [  0%] 0 of 1
[-        ] process > MAKE_GENOME_FILTER                 [  0%] 0 of 1
[67/8de9ff] process > FASTQC (control_R2_T1)             [  0%] 0 of 2
[30/61b27f] process > TRIMGALORE (control_R2_T1)         [ 50%] 1 of 2, failed: 1
[-        ] process > BWA_MEM                            -
[-        ] process > SORT_BAM                           -
[-        ] process > MERGED_LIB_BAM                     -
[-        ] process > MERGED_LIB_BAM_FILTER              -
[-        ] process > MERGED_LIB_BAM_REMOVE_ORPHAN       -
[-        ] process > MERGED_LIB_PRESEQ                  -
[-        ] process > MERGED_LIB_PICARD_METRICS          -
[-        ] process > MERGED_LIB_BIGWIG                  -
[-        ] process > MERGED_LIB_PLOTPROFILE             -
[-        ] process > MERGED_LIB_PLOTFINGERPRINT         -
[-        ] process > MERGED_LIB_MACS2                   -
[-        ] process > MERGED_LIB_MACS2_ANNOTATE          -
[-        ] process > MERGED_LIB_MACS2_QC                -
[-        ] process > MERGED_LIB_CONSENSUS               -
[-        ] process > MERGED_LIB_CONSENSUS_ANNOTATE      -
[-        ] process > MERGED_LIB_CONSENSUS_COUNTS        -
[-        ] process > MERGED_LIB_CONSENSUS_DESEQ2        -
[-        ] process > MERGED_LIB_ATAQV                   -
[-        ] process > MERGED_LIB_ATAQV_MKARV             -
[-        ] process > MERGED_REP_BAM                     -
[-        ] process > MERGED_REP_BIGWIG                  -
[-        ] process > MERGED_REP_MACS2                   -
[-        ] process > MERGED_REP_MACS2_ANNOTATE          -
[-        ] process > MERGED_REP_MACS2_QC                -
[-        ] process > MERGED_REP_CONSENSUS               -
[-        ] process > MERGED_REP_CONSENSUS_ANNOTATE      -
[-        ] process > MERGED_REP_CONSENSUS_COUNTS        -
[-        ] process > MERGED_REP_CONSENSUS_DESEQ2        -
[-        ] process > IGV                                -
[-        ] process > get_software_versions              [  0%] 0 of 1
[-        ] process > MULTIQC                            -
[-        ] process > output_documentation               [  0%] 0 of 1
Error executing process > 'TRIMGALORE (control_R2_T1)'

Caused by:
  Failed to submit process to grid scheduler for execution

Command executed:

  qsub -N nf-TRIMGALORE_c .command.run

Command exit status:
  32

Command output:
  qsub: 
   Job resource selection does not match any permitted configuration.
   Please review the job sizing guidance on:
   https://www.imperial.ac.uk/admin-services/ict/self-service/research-support/rcs/computing/

Work dir:
  /rds/general/user/bms20/ephemeral/tmp/30/61b27fee050798cf094085331202fe

Tip: view the complete command output by changing to the process work dir and entering the command `cat .command.out`
bschilder commented 3 years ago

Reply from Combiz

Could be due to the slightly different job sizing requirements of CX1 (main HPC) and medbio.. or updates to job sizing requirements from Imperial which happens from time to time. Usually the issue is the ratio of CPUs:RAM and for CX1 these are listed here: - https://www.imperial.ac.uk/admin-services/ict/self-service/research-support/rcs/computing/job-sizing-guidance/high-throughput/ The failing process (TRIMGALORE) is tagged with process_high and the corresponding job sizing params are from base.config here: - https://github.com/nf-core/atacseq/blob/1b3a832db5a53c92c2c89a4d6d79455e860461ad/conf/base.config#L33 My suggestion is to override this in your custom config (e.g. try reducing CPU from 12 to 8 for the job to be submitted to throughput or increase to 32 for general), e.g.: -

 withLabel:process_high {
    cpus = { check_max( 32 * task.attempt, 'cpus' ) }
    memory = { check_max( 62.GB * task.attempt, 'memory' ) }
    time = { check_max( 24.h * task.attempt, 'time' ) }
  }

When you get it working it would be good to share the new settings so the CX1 (https://github.com/nf-core/configs/blob/master/conf/imperial.config) and/or medbio (https://github.com/nf-core/configs/blob/master/conf/imperial_mb.config) institutional nf-core configs can be updated.

bschilder commented 3 years ago

Thanks so much @Combiz! I just gave it a try with withLabel:process_high ... got a bit further in some steps with this, but HPC still doesn't seem to like these job specs: Job resource selection does not match any permitted configuration. I can try playing around with this. Also got a bit further running via the medbio queue, but that also encountered some issues. But this seems to be related to the container? Which is weird, bc i already downloaded their singularity .sif file to avoid any issues with that.

-bash-4.2$ bash run_atacseq.sh 
N E X T F L O W  ~  version 20.10.0
Launching `nf-core/atacseq` [festering_carlsson] - revision: 1b3a832db5 [1.2.1]
----------------------------------------------------
                                        ,--./,-.
        ___     __   __   __   ___     /,-._.--~'
  |\ | |__  __ /  ` /  \ |__) |__         }  {
  | \| |       \__, \__/ |  \ |___     \`-._,-`-,
                                        `._,._,'
  nf-core/atacseq v1.2.1
----------------------------------------------------
Run Name             : festering_carlsson
Data Type            : Paired-End
Design File          : /rds/general/user/bms20/home/neurogenomics/GitRepos/CUT_n_TAG/raw_data/HK5M2BBXY/design_noindex2.csv
Genome               : GRCh37
Fasta File           : s3://ngi-igenomes/igenomes//Homo_sapiens/Ensembl/GRCh37/Sequence/WholeGenomeFasta/genome.fa
GTF File             : s3://ngi-igenomes/igenomes//Homo_sapiens/Ensembl/GRCh37/Annotation/Genes/genes.gtf
Gene BED File        : s3://ngi-igenomes/igenomes//Homo_sapiens/Ensembl/GRCh37/Annotation/Genes/genes.bed
BWA Index            : s3://ngi-igenomes/igenomes//Homo_sapiens/Ensembl/GRCh37/Sequence/BWAIndex/genome.fa
Blacklist BED        : /rds/general/user/bms20/home/.nextflow/assets/nf-core/atacseq/assets/blacklists/GRCh37-blacklist.bed
Mitochondrial Contig : MT
MACS2 Genome Size    : 2.7e9
Min Consensus Reps   : 1
MACS2 Narrow Peaks   : Yes
Trim R1              : 0 bp
Trim R2              : 0 bp
Trim 3' R1           : 0 bp
Trim 3' R2           : 0 bp
NextSeq Trim         : 0 bp
Fingerprint Bins     : 500000
Save Genome Index    : No
Max Resources        : 128 GB memory, 16 cpus, 10d time per job
Container            : singularity - /rds/general/user/bms20/home/atacseq_latest.sif
Output Dir           : /rds/general/user/bms20/home/neurogenomics/GitRepos/CUT_n_TAG/processed_data/HK5M2BBXY
Launch Dir           : /rds/general/project/neurogenomics-lab/live/GitRepos/CUT_n_TAG
Working Dir          : /rds/general/user/bms20/ephemeral/tmp
Script Dir           : /rds/general/user/bms20/home/.nextflow/assets/nf-core/atacseq
User                 : bms20
Config Profile       : standard
----------------------------------------------------
executor >  pbspro (3)
[57/a22311] process > CHECK_DESIGN (design_noindex2.csv) [100%] 1 of 1 ✔
[-        ] process > MAKE_TSS_BED                       -
[-        ] process > MAKE_GENOME_FILTER                 -
[df/45a206] process > FASTQC (control_R1_T1)             [  0%] 0 of 2
[-        ] process > TRIMGALORE                         [  0%] 0 of 2
[-        ] process > BWA_MEM                            -
[-        ] process > SORT_BAM                           -
[-        ] process > MERGED_LIB_BAM                     -
[-        ] process > MERGED_LIB_BAM_FILTER              -
[-        ] process > MERGED_LIB_BAM_REMOVE_ORPHAN       -
[-        ] process > MERGED_LIB_PRESEQ                  -
[-        ] process > MERGED_LIB_PICARD_METRICS          -
[-        ] process > MERGED_LIB_BIGWIG                  -
[-        ] process > MERGED_LIB_PLOTPROFILE             -
[-        ] process > MERGED_LIB_PLOTFINGERPRINT         -
[-        ] process > MERGED_LIB_MACS2                   -
[-        ] process > MERGED_LIB_MACS2_ANNOTATE          -
[-        ] process > MERGED_LIB_MACS2_QC                -
[-        ] process > MERGED_LIB_CONSENSUS               -
executor >  pbspro (3)
[57/a22311] process > CHECK_DESIGN (design_noindex2.csv) [100%] 1 of 1 ✔
[-        ] process > MAKE_TSS_BED                       -
[-        ] process > MAKE_GENOME_FILTER                 -
[df/45a206] process > FASTQC (control_R1_T1)             [  0%] 0 of 2
[f4/7b7908] process > TRIMGALORE (control_R2_T1)         [ 50%] 1 of 2, failed: 1
[-        ] process > BWA_MEM                            -
[-        ] process > SORT_BAM                           -
[-        ] process > MERGED_LIB_BAM                     -
[-        ] process > MERGED_LIB_BAM_FILTER              -
[-        ] process > MERGED_LIB_BAM_REMOVE_ORPHAN       -
[-        ] process > MERGED_LIB_PRESEQ                  -
[-        ] process > MERGED_LIB_PICARD_METRICS          -
[-        ] process > MERGED_LIB_BIGWIG                  -
[-        ] process > MERGED_LIB_PLOTPROFILE             -
[-        ] process > MERGED_LIB_PLOTFINGERPRINT         -
[-        ] process > MERGED_LIB_MACS2                   -
[-        ] process > MERGED_LIB_MACS2_ANNOTATE          -
[-        ] process > MERGED_LIB_MACS2_QC                -
[-        ] process > MERGED_LIB_CONSENSUS               -
[-        ] process > MERGED_LIB_CONSENSUS_ANNOTATE      -
executor >  pbspro (3)
[57/a22311] process > CHECK_DESIGN (design_noindex2.csv) [100%] 1 of 1 ✔
[-        ] process > MAKE_TSS_BED                       [  0%] 0 of 1
[-        ] process > MAKE_GENOME_FILTER                 [  0%] 0 of 1
[df/45a206] process > FASTQC (control_R1_T1)             [  0%] 0 of 2
[f4/7b7908] process > TRIMGALORE (control_R2_T1)         [ 50%] 1 of 2, failed: 1
[-        ] process > BWA_MEM                            -
[-        ] process > SORT_BAM                           -
[-        ] process > MERGED_LIB_BAM                     -
[-        ] process > MERGED_LIB_BAM_FILTER              -
[-        ] process > MERGED_LIB_BAM_REMOVE_ORPHAN       -
[-        ] process > MERGED_LIB_PRESEQ                  -
[-        ] process > MERGED_LIB_PICARD_METRICS          -
[-        ] process > MERGED_LIB_BIGWIG                  -
[-        ] process > MERGED_LIB_PLOTPROFILE             -
[-        ] process > MERGED_LIB_PLOTFINGERPRINT         -
[-        ] process > MERGED_LIB_MACS2                   -
[-        ] process > MERGED_LIB_MACS2_ANNOTATE          -
[-        ] process > MERGED_LIB_MACS2_QC                -
[-        ] process > MERGED_LIB_CONSENSUS               -
[-        ] process > MERGED_LIB_CONSENSUS_ANNOTATE      -
executor >  pbspro (3)
[57/a22311] process > CHECK_DESIGN (design_noindex2.csv) [100%] 1 of 1 ✔
[-        ] process > MAKE_TSS_BED                       [  0%] 0 of 1
[-        ] process > MAKE_GENOME_FILTER                 [  0%] 0 of 1
[df/45a206] process > FASTQC (control_R1_T1)             [  0%] 0 of 2
[f4/7b7908] process > TRIMGALORE (control_R2_T1)         [ 50%] 1 of 2, failed: 1
[-        ] process > BWA_MEM                            -
[-        ] process > SORT_BAM                           -
[-        ] process > MERGED_LIB_BAM                     -
[-        ] process > MERGED_LIB_BAM_FILTER              -
[-        ] process > MERGED_LIB_BAM_REMOVE_ORPHAN       -
[-        ] process > MERGED_LIB_PRESEQ                  -
[-        ] process > MERGED_LIB_PICARD_METRICS          -
[-        ] process > MERGED_LIB_BIGWIG                  -
[-        ] process > MERGED_LIB_PLOTPROFILE             -
[-        ] process > MERGED_LIB_PLOTFINGERPRINT         -
[-        ] process > MERGED_LIB_MACS2                   -
[-        ] process > MERGED_LIB_MACS2_ANNOTATE          -
[-        ] process > MERGED_LIB_MACS2_QC                -
[-        ] process > MERGED_LIB_CONSENSUS               -
[-        ] process > MERGED_LIB_CONSENSUS_ANNOTATE      -
executor >  pbspro (3)
[57/a22311] process > CHECK_DESIGN (design_noindex2.csv) [100%] 1 of 1 ✔
[-        ] process > MAKE_TSS_BED                       [  0%] 0 of 1
[-        ] process > MAKE_GENOME_FILTER                 [  0%] 0 of 1
[a5/677464] process > FASTQC (control_R2_T1)             [ 33%] 1 of 3, failed: 1, retries: 1
[f4/7b7908] process > TRIMGALORE (control_R2_T1)         [ 50%] 1 of 2, failed: 1
[-        ] process > BWA_MEM                            -
[-        ] process > SORT_BAM                           -
[-        ] process > MERGED_LIB_BAM                     -
[-        ] process > MERGED_LIB_BAM_FILTER              -
[-        ] process > MERGED_LIB_BAM_REMOVE_ORPHAN       -
[-        ] process > MERGED_LIB_PRESEQ                  -
[-        ] process > MERGED_LIB_PICARD_METRICS          -
[-        ] process > MERGED_LIB_BIGWIG                  -
[-        ] process > MERGED_LIB_PLOTPROFILE             -
[-        ] process > MERGED_LIB_PLOTFINGERPRINT         -
[-        ] process > MERGED_LIB_MACS2                   -
[-        ] process > MERGED_LIB_MACS2_ANNOTATE          -
[-        ] process > MERGED_LIB_MACS2_QC                -
[-        ] process > MERGED_LIB_CONSENSUS               -
[-        ] process > MERGED_LIB_CONSENSUS_ANNOTATE      -
[-        ] process > MERGED_LIB_CONSENSUS_COUNTS        -
[-        ] process > MERGED_LIB_CONSENSUS_DESEQ2        -
[-        ] process > MERGED_LIB_ATAQV                   -
[-        ] process > MERGED_LIB_ATAQV_MKARV             -
[-        ] process > MERGED_REP_BAM                     -
[-        ] process > MERGED_REP_BIGWIG                  -
[-        ] process > MERGED_REP_MACS2                   -
[-        ] process > MERGED_REP_MACS2_ANNOTATE          -
[-        ] process > MERGED_REP_MACS2_QC                -
[-        ] process > MERGED_REP_CONSENSUS               -
[-        ] process > MERGED_REP_CONSENSUS_ANNOTATE      -
[-        ] process > MERGED_REP_CONSENSUS_COUNTS        -
[-        ] process > MERGED_REP_CONSENSUS_DESEQ2        -
[-        ] process > IGV                                -
[-        ] process > get_software_versions              [  0%] 0 of 1
[-        ] process > MULTIQC                            -
[-        ] process > output_documentation               [  0%] 0 of 1
[a5/677464] NOTE: Process `FASTQC (control_R2_T1)` terminated with an error exit status (134) -- Execution is retried (1)
Error executing process > 'TRIMGALORE (control_R2_T1)'

Caused by:
  Failed to submit process to grid scheduler for execution

Command executed:

  qsub -N nf-TRIMGALORE_c .command.run

Command exit status:
  32

Command output:
  qsub: 
   Job resource selection does not match any permitted configuration.
   Please review the job sizing guidance on:
   https://www.imperial.ac.uk/admin-services/ict/self-service/research-support/rcs/computing/

Work dir:
  /rds/general/user/bms20/ephemeral/tmp/f4/7b7908c4ab230ce46e04cfac9c35ab

Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named `.command.sh`
bschilder commented 3 years ago

Narun's reply

container creation failing is an intermittent issue on our HPC. You might just need to delete your workdir from ephemeral and run the pipeline again

bschilder commented 3 years ago

that seems to be helping a lot! thanks!!