search

epi2me-labs / wf-bacterial-genomes

Small variant calling for haploid samples
https://labs.epi2me.io/
Other
25 stars 8 forks source link

Help? Error running nextflow run epi2me-labs/wf-bacterial-genomes –-help #39

Closed greytear closed 3 months ago

greytear commented 3 months ago

Ask away!

While trying to assist a user to run wf-bacterial-genomes on a Linux Cluster, I installed nextflow. After installation,

i tried wf-bacterial-genomes --help and got an error:

./nextflow run epi2me-labs/wf-bacterial-genomes –-help

N E X T F L O W ~ version 24.04.2

Launching https://github.com/epi2me-labs/wf-bacterial-genomes [focused_marconi] DSL2 - revision: 6af54574c3 [master]

ERROR ~ Script compilation error

1 error

on .nextflow.log i got a mention to the same error

Jul-03 17:28:33.365 [main] DEBUG nextflow.script.ScriptRunner - Parsed script files: Jul-03 17:28:33.374 [main] ERROR nextflow.cli.Launcher - Script compilation error

1 error

Could you please provide some help on solving this? Thanks!

julibeg commented 3 months ago

Hi @greytear , Nextflow introduced a new feature in v24.04 that conflicts with some of our workflows. This will be fixed in the next wf-bacterial-genomes release. In the meantime, you can run the workflow with Nextflow version 23.10 by running it as

NXF_VER=23.10.1 nextflow run epi2me-labs/wf-bacterial-genomes ...

Please let us know if this doesn't resolve the issue. Thanks for your patience!

greytear commented 3 months ago

Thank You, @julibeg ! Athough that seemed to solve part of the issue, it stills throws errors. For example, 

NXF_VER=23.10.1 ./nextflow run epi2me-labs/wf-bacterial-genomes –-help
Nextflow 24.04.2 is available - Please consider updating your version to it
N E X T F L O W  ~  version 23.10.1
Launching `https://github.com/epi2me-labs/wf-bacterial-genomes` [naughty_lumiere] DSL2 - revision: 6af54574c3 [master]
WARN: NEXTFLOW RECURSION IS A PREVIEW FEATURE - SYNTAX AND FUNCTIONALITY CAN CHANGE IN FUTURE RELEASES

ERROR ~ Validation of pipeline parameters failed!

 -- Check '.nextflow.log' file for details
ERROR ~ * Missing required parameter: --fastq

I disregarded this one, and tried to run the workflow with the demo dataset. For the sake of clarity, we are using singularity, here, so I passed that to the -profile option:

NXF_VER=23.10.1 ./nextflow run epi2me-labs/wf-bacterial-genomes --fastq wf-bacterial-genomes-demo/isolates_fastq     --isolates     --sample_sheet wf-bacterial-genomes-demo/isolates_sample_sheet.csv -profile singularity
Nextflow 24.04.2 is available - Please consider updating your version to it
N E X T F L O W  ~  version 23.10.1
Launching `https://github.com/epi2me-labs/wf-bacterial-genomes` [modest_borg] DSL2 - revision: 6af54574c3 [master]
WARN: NEXTFLOW RECURSION IS A PREVIEW FEATURE - SYNTAX AND FUNCTIONALITY CAN CHANGE IN FUTURE RELEASES

||||||||||   _____ ____ ___ ____  __  __ _____      _       _
||||||||||  | ____|  _ \_ _|___ \|  \/  | ____|    | | __ _| |__  ___
|||||       |  _| | |_) | |  __) | |\/| |  _| _____| |/ _` | '_ \/ __|
|||||       | |___|  __/| | / __/| |  | | |__|_____| | (_| | |_) \__ \
||||||||||  |_____|_|  |___|_____|_|  |_|_____|    |_|\__,_|_.__/|___/
||||||||||  wf-bacterial-genomes v1.2.0-g6af5457
--------------------------------------------------------------------------------
Core Nextflow options
  revision       : master
  runName        : modest_borg
  containerEngine: singularity
  container      : [withLabel:wf_common:ontresearch/wf-common:sha1c5febff9f75143710826498b093d9769a5edbb9, withLabel:wfbacterialgenomes:ontresearch/wf-bacterial-genomes:shaa4cda1aeeda01242c54f4af03419e9623397dc0c, withLabel:prokka:ontresearch/prokka:sha08669655982fbef7c750c7895e97e100196c4967, withLabel:medaka:ontresearch/medaka:sha4f35e9f6f231dc8343c98e04d43517d4a38528a2, withLabel:amr:genomicepidemiology/resfinder:4.3.2, withLabel:mlst:staphb/mlst:2.23.0, withLabel:seqsero2:ontresearch/seqsero2:sha96053b39b281e404cf1cf9c4684fa7dbc4e2761d]
  launchDir      : /mnt/porao/HOMERMT/xxxx
  workDir        : /mnt/porao/HOMERMT/xxxx/work
  projectDir     : /home/xxxx/.nextflow/assets/epi2me-labs/wf-bacterial-genomes
  userName       : xxxx
  profile        : singularity
  configFiles    : /home/xxxx/.nextflow/assets/epi2me-labs/wf-bacterial-genomes/nextflow.config

Input Options
  fastq          : wf-bacterial-genomes-demo/isolates_fastq

Sample Options
  sample_sheet   : wf-bacterial-genomes-demo/isolates_sample_sheet.csv

Isolate options
  isolates       : true

!! Only displaying parameters that differ from the pipeline defaults !!
--------------------------------------------------------------------------------
If you use epi2me-labs/wf-bacterial-genomes for your analysis please cite:

* The nf-core framework
  https://doi.org/10.1038/s41587-020-0439-x

--------------------------------------------------------------------------------
This is epi2me-labs/wf-bacterial-genomes v1.2.0-g6af5457.
--------------------------------------------------------------------------------
Searching input for [.fastq, .fastq.gz, .fq, .fq.gz] files.
[-        ] process > validate_sample_sheet                               -
[-        ] process > fastcat                                             -
[-        ] process > calling_pipeline:ingressCheckpoint                  -
[-        ] process > calling_pipeline:deNovo                             -
[-        ] process > validate_sample_sheet                               -
[-        ] process > fastcat                                             -
[-        ] process > calling_pipeline:ingressCheckpoint                  -
[-        ] process > calling_pipeline:deNovo                             -
[-        ] process > calling_pipeline:alignReads                         -
[-        ] process > calling_pipeline:alignmentCheckpoint                -
[-        ] process > calling_pipeline:readStats                          -
[-        ] process > calling_pipeline:coverStats                         -
[-        ] process > calling_pipeline:splitRegions                       -
[-        ] process > calling_pipeline:lookup_medaka_consensus_model      -
[-        ] process > calling_pipeline:lookup_medaka_variant_model        -
[-        ] process > calling_pipeline:medakaNetwork                      -
[-        ] process > calling_pipeline:medakaConsensus                    -
[-        ] process > calling_pipeline:assemblyCheckpoint                 -
[-        ] process > calling_pipeline:variantCheckpoint                  -
[-        ] process > calling_pipeline:runProkka                          -
[-        ] process > calling_pipeline:annotationCheckpoint               -
[-        ] process > calling_pipeline:run_isolates:mlstSearch            -
[-        ] process > calling_pipeline:run_isolates:getPointfinderSpecies -
[-        ] process > calling_pipeline:run_isolates:resfinder             -
[-        ] process > calling_pipeline:run_isolates:processResfinder      -
[-        ] process > calling_pipeline:run_isolates:serotyping            -
[-        ] process > calling_pipeline:amrCheckpoint                      -
[-        ] process > calling_pipeline:prokkaVersion                      -
[-        ] process > calling_pipeline:medakaVersion                      -
[-        ] process > calling_pipeline:mlstVersion                        -
[-        ] process > calling_pipeline:getVersions                        -
[-        ] process > calling_pipeline:getParams                          -
[-        ] process > calling_pipeline:collect_results                    -
[-        ] process > calling_pipeline:createRunModel                     -
[-        ] process > calling_pipeline:makeReport                         -
[-        ] process > calling_pipeline:reportingCheckpoint                -
[-        ] process > calling_pipeline:makePerSampleReports               -
[-        ] process > calling_pipeline:perSampleReportingCheckpoint       -
[-        ] process > calling_pipeline:accumulateCheckpoints              -
[-        ] process > calling_pipeline:collectFastqIngressResultsInDir    -
[-        ] process > output                                              -
Running Denovo assembly.
Pulling Singularity image docker://ontresearch/wf-bacterial-genomes:shaa4cda1aeeda01242c54f4af03419e9623397dc0c [cache /mnt/porao/HOMERMT/xxxx/work/singularity/ontresearch-wf-bacterial-genomes-shaa4cda1aeeda01242c54f4af03419e9623397dc0c.img]
[-        ] process > validate_sample_sheet                               -
[-        ] process > fastcat                                             -
[-        ] process > calling_pipeline:ingressCheckpoint                  -
[-        ] process > calling_pipeline:deNovo                             -
[-        ] process > calling_pipeline:alignReads                         -
[-        ] process > calling_pipeline:alignmentCheckpoint                -
[-        ] process > calling_pipeline:readStats                          -
[-        ] process > calling_pipeline:coverStats                         -
[-        ] process > calling_pipeline:splitRegions                       -
[-        ] process > calling_pipeline:lookup_medaka_consensus_model      -
[-        ] process > calling_pipeline:lookup_medaka_variant_model        -
[-        ] process > calling_pipeline:medakaNetwork                      -
[-        ] process > calling_pipeline:medakaConsensus                    -
[-        ] process > calling_pipeline:assemblyCheckpoint                 -
[-        ] process > calling_pipeline:variantCheckpoint                  -
[-        ] process > calling_pipeline:runProkka                          -
[-        ] process > calling_pipeline:annotationCheckpoint               -
[-        ] process > calling_pipeline:run_isolates:mlstSearch            -
[-        ] process > calling_pipeline:run_isolates:getPointfinderSpecies -
[-        ] process > calling_pipeline:run_isolates:resfinder             -
[-        ] process > calling_pipeline:run_isolates:processResfinder      -
[-        ] process > calling_pipeline:run_isolates:serotyping            -
[-        ] process > calling_pipeline:amrCheckpoint                      -
[-        ] process > calling_pipeline:prokkaVersion                      -
[-        ] process > calling_pipeline:medakaVersion                      -
[-        ] process > calling_pipeline:mlstVersion                        -
[-        ] process > calling_pipeline:getVersions                        -
[-        ] process > calling_pipeline:getParams                          -
[-        ] process > calling_pipeline:collect_results                    -
[-        ] process > calling_pipeline:createRunModel                     -
[-        ] process > calling_pipeline:makeReport                         -
[-        ] process > calling_pipeline:reportingCheckpoint                -
[-        ] process > calling_pipeline:makePerSampleReports               -
[-        ] process > calling_pipeline:perSampleReportingCheckpoint       -
[-        ] process > calling_pipeline:accumulateCheckpoints              -
[-        ] process > calling_pipeline:collectFastqIngressResultsInDir    -
[-        ] process > output                                              -
Running Denovo assembly.
Pulling Singularity image docker://ontresearch/wf-bacterial-genomes:shaa4cda1aeeda01242c54f4af03419e9623397dc0c [cache /mnt/porao/HOMERMT/xxxx/work/singularity/ontresearch-wf-bacterial-genomes-shaa4cda1aeeda01242c54f4af03419e9623397dc0c.img]
Pulling Singularity image docker://ontresearch/prokka:sha08669655982fbef7c750c7895e97e100196c4967 [cache /mnt/porao/HOMERMT/xxxx/work/singularity/ontresearch-prokka-sha08669655982fbef7c750c7895e97e100196c4967.img]
Pulling Singularity image docker://ontresearch/wf-common:sha1c5febff9f75143710826498b093d9769a5edbb9 [cache /mnt/porao/HOMERMT/xxxx/work/singularity/ontresearch-wf-common-sha1c5febff9f75143710826498b093d9769a5edbb9.img]
WARN: Singularity cache directory has not been defined -- Remote image will be stored in the path: /mnt/porao/HOMERMT/xxxx/work/singularity -- Use the environment variable NXF_SINGULARITY_CACHEDIR to specify a different location
ERROR ~ Error executing process > 'calling_pipeline:lookup_medaka_consensus_model (1)'

Caused by:
  Failed to pull singularity image
  command: singularity pull  --name ontresearch-wf-bacterial-genomes-shaa4cda1aeeda01242c54f4af03419e9623397dc0c.img.pulling.1720079110322 docker://ontresearch/wf-bacterial-genomes:shaa4cda1aeeda01242c54f4af03419e9623397dc0c > /dev/null
  status : 255
  message:
    INFO:    Converting OCI blobs to SIF format
    INFO:    Starting build...
    Getting image source signatures
    Copying blob sha256:edaedc954fb53f42a7754a6e2d1b57f091bc9b11063bc445c2e325ea448f8f68
    Copying blob sha256:0456c71d8011447360dd461b49b6a1a4cb36edf8f7250817e99ceeb0d353f06b
    Copying blob sha256:38cad909cfed9fb07cc2c6612612435d52302b48ebcf94a2a4b77e02b8414142
    Copying blob sha256:88397e6fd840152eca6643094559385065a6ab98a1c3c7e75eb676b9d97aad97
    Copying blob sha256:e8b63ef4ca0ca489bb848ecf9f4db232528d222a2c0bb2a599b312ce6b5b551a
    Copying blob sha256:f9432144bf11a2ae3ffb7360b4e32f7b18c75c7c8c125274ec3cf49d0de28e24
    Copying blob sha256:cd889bc730171a0013130be60f28f869d7e7cc4d923396628660de9b67f7eec8
    Copying blob sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1
    Copying blob sha256:2042b0824681af825bf6a3c7d48cf91a218427db24a26198aaa7a2e92145081d
    Copying blob sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1
    Copying blob sha256:973691ee7cdac427dd9005f99675e8e77fb7feda0e94e11626a95e46e5d2973e
    Copying blob sha256:4a9d7e589e1778374b42cbdb32d978af00de957133897cdc1171659808fb6f7f
    Copying blob sha256:3fc3949421420f916ca77ede3295a7f762d493b84b0bc2e4a2212f8573b76e3c
    Copying blob sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1
    Copying blob sha256:6de220563c5cae03fae0f6b66c78c7e1091b6d2a91b0d199ec712fd216399bc6
    Copying blob sha256:7f4c1460c3a6a5c0a29dd740394343f9e4dc867062c299febc85015eb3cf8606
    Copying blob sha256:8e13bd657a727a4c167b0e27b05aa3ee5dab63fda01f8b2643400f4c10053dd8
    Copying blob sha256:d8af254b567f3b0a15ba645cf6dee25e3223b4a610fe237a3ad31a4dedf679fc
    Copying blob sha256:786231868c62c95197f37ffa3046b4f0ccffc11bd49a4f206a032c700eb18ef6
    Copying blob sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1
    Copying config sha256:755ac014996a07c3bf534514c4b2e0f7c0b97a0985a8be3111b8d76f22d8c30c
    Writing manifest to image destination
    Storing signatures
    FATAL:   While making image from oci registry: error fetching image to cache: while building SIF from layers: conveyor failed to get: no descriptor found for reference "cf10adfeb53aa11be154fde8365c9c19854952c384b359d19fcc9d323c3b32c8"

 -- Check '.nextflow.log' file for details

Is there any problem with the singularity/docker repository? It seems to me that the workflow is failing while building the SIF image....

Many thanks for your help!

julibeg commented 3 months ago

Hi @greytear,

Regarding the --help command:

It looks like somehow an en-dash made it into the --help in the README (instead of two hyphens; it's –-help but should be --help). If you copy + paste the below, it should work

NXF_VER=23.10.1 ./nextflow run epi2me-labs/wf-bacterial-genomes --help

The README will be updated in the next release.

Regarding the Singularity issue:

This could be an intermittent connectivity issue. Could you please try again and confirm that you get the same error? Thanks!

greytear commented 3 months ago

Thanks, @julibeg ! It seemed to be, indeed, an intermitent issue. Reissued the command and got it working.

Many thanks for the fabulous support!

julibeg commented 3 months ago

Happy to help!