Closed greytear closed 3 months ago
Hi @greytear , Nextflow introduced a new feature in v24.04 that conflicts with some of our workflows. This will be fixed in the next wf-bacterial-genomes release. In the meantime, you can run the workflow with Nextflow version 23.10 by running it as
NXF_VER=23.10.1 nextflow run epi2me-labs/wf-bacterial-genomes ...
Please let us know if this doesn't resolve the issue. Thanks for your patience!
Thank You, @julibeg ! Athough that seemed to solve part of the issue, it stills throws errors. For example,
NXF_VER=23.10.1 ./nextflow run epi2me-labs/wf-bacterial-genomes –-help
Nextflow 24.04.2 is available - Please consider updating your version to it
N E X T F L O W ~ version 23.10.1
Launching `https://github.com/epi2me-labs/wf-bacterial-genomes` [naughty_lumiere] DSL2 - revision: 6af54574c3 [master]
WARN: NEXTFLOW RECURSION IS A PREVIEW FEATURE - SYNTAX AND FUNCTIONALITY CAN CHANGE IN FUTURE RELEASES
ERROR ~ Validation of pipeline parameters failed!
-- Check '.nextflow.log' file for details
ERROR ~ * Missing required parameter: --fastq
I disregarded this one, and tried to run the workflow with the demo dataset. For the sake of clarity, we are using singularity, here, so I passed that to the -profile option:
NXF_VER=23.10.1 ./nextflow run epi2me-labs/wf-bacterial-genomes --fastq wf-bacterial-genomes-demo/isolates_fastq --isolates --sample_sheet wf-bacterial-genomes-demo/isolates_sample_sheet.csv -profile singularity
Nextflow 24.04.2 is available - Please consider updating your version to it
N E X T F L O W ~ version 23.10.1
Launching `https://github.com/epi2me-labs/wf-bacterial-genomes` [modest_borg] DSL2 - revision: 6af54574c3 [master]
WARN: NEXTFLOW RECURSION IS A PREVIEW FEATURE - SYNTAX AND FUNCTIONALITY CAN CHANGE IN FUTURE RELEASES
|||||||||| _____ ____ ___ ____ __ __ _____ _ _
|||||||||| | ____| _ \_ _|___ \| \/ | ____| | | __ _| |__ ___
||||| | _| | |_) | | __) | |\/| | _| _____| |/ _` | '_ \/ __|
||||| | |___| __/| | / __/| | | | |__|_____| | (_| | |_) \__ \
|||||||||| |_____|_| |___|_____|_| |_|_____| |_|\__,_|_.__/|___/
|||||||||| wf-bacterial-genomes v1.2.0-g6af5457
--------------------------------------------------------------------------------
Core Nextflow options
revision : master
runName : modest_borg
containerEngine: singularity
container : [withLabel:wf_common:ontresearch/wf-common:sha1c5febff9f75143710826498b093d9769a5edbb9, withLabel:wfbacterialgenomes:ontresearch/wf-bacterial-genomes:shaa4cda1aeeda01242c54f4af03419e9623397dc0c, withLabel:prokka:ontresearch/prokka:sha08669655982fbef7c750c7895e97e100196c4967, withLabel:medaka:ontresearch/medaka:sha4f35e9f6f231dc8343c98e04d43517d4a38528a2, withLabel:amr:genomicepidemiology/resfinder:4.3.2, withLabel:mlst:staphb/mlst:2.23.0, withLabel:seqsero2:ontresearch/seqsero2:sha96053b39b281e404cf1cf9c4684fa7dbc4e2761d]
launchDir : /mnt/porao/HOMERMT/xxxx
workDir : /mnt/porao/HOMERMT/xxxx/work
projectDir : /home/xxxx/.nextflow/assets/epi2me-labs/wf-bacterial-genomes
userName : xxxx
profile : singularity
configFiles : /home/xxxx/.nextflow/assets/epi2me-labs/wf-bacterial-genomes/nextflow.config
Input Options
fastq : wf-bacterial-genomes-demo/isolates_fastq
Sample Options
sample_sheet : wf-bacterial-genomes-demo/isolates_sample_sheet.csv
Isolate options
isolates : true
!! Only displaying parameters that differ from the pipeline defaults !!
--------------------------------------------------------------------------------
If you use epi2me-labs/wf-bacterial-genomes for your analysis please cite:
* The nf-core framework
https://doi.org/10.1038/s41587-020-0439-x
--------------------------------------------------------------------------------
This is epi2me-labs/wf-bacterial-genomes v1.2.0-g6af5457.
--------------------------------------------------------------------------------
Searching input for [.fastq, .fastq.gz, .fq, .fq.gz] files.
[- ] process > validate_sample_sheet -
[- ] process > fastcat -
[- ] process > calling_pipeline:ingressCheckpoint -
[- ] process > calling_pipeline:deNovo -
[- ] process > validate_sample_sheet -
[- ] process > fastcat -
[- ] process > calling_pipeline:ingressCheckpoint -
[- ] process > calling_pipeline:deNovo -
[- ] process > calling_pipeline:alignReads -
[- ] process > calling_pipeline:alignmentCheckpoint -
[- ] process > calling_pipeline:readStats -
[- ] process > calling_pipeline:coverStats -
[- ] process > calling_pipeline:splitRegions -
[- ] process > calling_pipeline:lookup_medaka_consensus_model -
[- ] process > calling_pipeline:lookup_medaka_variant_model -
[- ] process > calling_pipeline:medakaNetwork -
[- ] process > calling_pipeline:medakaConsensus -
[- ] process > calling_pipeline:assemblyCheckpoint -
[- ] process > calling_pipeline:variantCheckpoint -
[- ] process > calling_pipeline:runProkka -
[- ] process > calling_pipeline:annotationCheckpoint -
[- ] process > calling_pipeline:run_isolates:mlstSearch -
[- ] process > calling_pipeline:run_isolates:getPointfinderSpecies -
[- ] process > calling_pipeline:run_isolates:resfinder -
[- ] process > calling_pipeline:run_isolates:processResfinder -
[- ] process > calling_pipeline:run_isolates:serotyping -
[- ] process > calling_pipeline:amrCheckpoint -
[- ] process > calling_pipeline:prokkaVersion -
[- ] process > calling_pipeline:medakaVersion -
[- ] process > calling_pipeline:mlstVersion -
[- ] process > calling_pipeline:getVersions -
[- ] process > calling_pipeline:getParams -
[- ] process > calling_pipeline:collect_results -
[- ] process > calling_pipeline:createRunModel -
[- ] process > calling_pipeline:makeReport -
[- ] process > calling_pipeline:reportingCheckpoint -
[- ] process > calling_pipeline:makePerSampleReports -
[- ] process > calling_pipeline:perSampleReportingCheckpoint -
[- ] process > calling_pipeline:accumulateCheckpoints -
[- ] process > calling_pipeline:collectFastqIngressResultsInDir -
[- ] process > output -
Running Denovo assembly.
Pulling Singularity image docker://ontresearch/wf-bacterial-genomes:shaa4cda1aeeda01242c54f4af03419e9623397dc0c [cache /mnt/porao/HOMERMT/xxxx/work/singularity/ontresearch-wf-bacterial-genomes-shaa4cda1aeeda01242c54f4af03419e9623397dc0c.img]
[- ] process > validate_sample_sheet -
[- ] process > fastcat -
[- ] process > calling_pipeline:ingressCheckpoint -
[- ] process > calling_pipeline:deNovo -
[- ] process > calling_pipeline:alignReads -
[- ] process > calling_pipeline:alignmentCheckpoint -
[- ] process > calling_pipeline:readStats -
[- ] process > calling_pipeline:coverStats -
[- ] process > calling_pipeline:splitRegions -
[- ] process > calling_pipeline:lookup_medaka_consensus_model -
[- ] process > calling_pipeline:lookup_medaka_variant_model -
[- ] process > calling_pipeline:medakaNetwork -
[- ] process > calling_pipeline:medakaConsensus -
[- ] process > calling_pipeline:assemblyCheckpoint -
[- ] process > calling_pipeline:variantCheckpoint -
[- ] process > calling_pipeline:runProkka -
[- ] process > calling_pipeline:annotationCheckpoint -
[- ] process > calling_pipeline:run_isolates:mlstSearch -
[- ] process > calling_pipeline:run_isolates:getPointfinderSpecies -
[- ] process > calling_pipeline:run_isolates:resfinder -
[- ] process > calling_pipeline:run_isolates:processResfinder -
[- ] process > calling_pipeline:run_isolates:serotyping -
[- ] process > calling_pipeline:amrCheckpoint -
[- ] process > calling_pipeline:prokkaVersion -
[- ] process > calling_pipeline:medakaVersion -
[- ] process > calling_pipeline:mlstVersion -
[- ] process > calling_pipeline:getVersions -
[- ] process > calling_pipeline:getParams -
[- ] process > calling_pipeline:collect_results -
[- ] process > calling_pipeline:createRunModel -
[- ] process > calling_pipeline:makeReport -
[- ] process > calling_pipeline:reportingCheckpoint -
[- ] process > calling_pipeline:makePerSampleReports -
[- ] process > calling_pipeline:perSampleReportingCheckpoint -
[- ] process > calling_pipeline:accumulateCheckpoints -
[- ] process > calling_pipeline:collectFastqIngressResultsInDir -
[- ] process > output -
Running Denovo assembly.
Pulling Singularity image docker://ontresearch/wf-bacterial-genomes:shaa4cda1aeeda01242c54f4af03419e9623397dc0c [cache /mnt/porao/HOMERMT/xxxx/work/singularity/ontresearch-wf-bacterial-genomes-shaa4cda1aeeda01242c54f4af03419e9623397dc0c.img]
Pulling Singularity image docker://ontresearch/prokka:sha08669655982fbef7c750c7895e97e100196c4967 [cache /mnt/porao/HOMERMT/xxxx/work/singularity/ontresearch-prokka-sha08669655982fbef7c750c7895e97e100196c4967.img]
Pulling Singularity image docker://ontresearch/wf-common:sha1c5febff9f75143710826498b093d9769a5edbb9 [cache /mnt/porao/HOMERMT/xxxx/work/singularity/ontresearch-wf-common-sha1c5febff9f75143710826498b093d9769a5edbb9.img]
WARN: Singularity cache directory has not been defined -- Remote image will be stored in the path: /mnt/porao/HOMERMT/xxxx/work/singularity -- Use the environment variable NXF_SINGULARITY_CACHEDIR to specify a different location
ERROR ~ Error executing process > 'calling_pipeline:lookup_medaka_consensus_model (1)'
Caused by:
Failed to pull singularity image
command: singularity pull --name ontresearch-wf-bacterial-genomes-shaa4cda1aeeda01242c54f4af03419e9623397dc0c.img.pulling.1720079110322 docker://ontresearch/wf-bacterial-genomes:shaa4cda1aeeda01242c54f4af03419e9623397dc0c > /dev/null
status : 255
message:
INFO: Converting OCI blobs to SIF format
INFO: Starting build...
Getting image source signatures
Copying blob sha256:edaedc954fb53f42a7754a6e2d1b57f091bc9b11063bc445c2e325ea448f8f68
Copying blob sha256:0456c71d8011447360dd461b49b6a1a4cb36edf8f7250817e99ceeb0d353f06b
Copying blob sha256:38cad909cfed9fb07cc2c6612612435d52302b48ebcf94a2a4b77e02b8414142
Copying blob sha256:88397e6fd840152eca6643094559385065a6ab98a1c3c7e75eb676b9d97aad97
Copying blob sha256:e8b63ef4ca0ca489bb848ecf9f4db232528d222a2c0bb2a599b312ce6b5b551a
Copying blob sha256:f9432144bf11a2ae3ffb7360b4e32f7b18c75c7c8c125274ec3cf49d0de28e24
Copying blob sha256:cd889bc730171a0013130be60f28f869d7e7cc4d923396628660de9b67f7eec8
Copying blob sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1
Copying blob sha256:2042b0824681af825bf6a3c7d48cf91a218427db24a26198aaa7a2e92145081d
Copying blob sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1
Copying blob sha256:973691ee7cdac427dd9005f99675e8e77fb7feda0e94e11626a95e46e5d2973e
Copying blob sha256:4a9d7e589e1778374b42cbdb32d978af00de957133897cdc1171659808fb6f7f
Copying blob sha256:3fc3949421420f916ca77ede3295a7f762d493b84b0bc2e4a2212f8573b76e3c
Copying blob sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1
Copying blob sha256:6de220563c5cae03fae0f6b66c78c7e1091b6d2a91b0d199ec712fd216399bc6
Copying blob sha256:7f4c1460c3a6a5c0a29dd740394343f9e4dc867062c299febc85015eb3cf8606
Copying blob sha256:8e13bd657a727a4c167b0e27b05aa3ee5dab63fda01f8b2643400f4c10053dd8
Copying blob sha256:d8af254b567f3b0a15ba645cf6dee25e3223b4a610fe237a3ad31a4dedf679fc
Copying blob sha256:786231868c62c95197f37ffa3046b4f0ccffc11bd49a4f206a032c700eb18ef6
Copying blob sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1
Copying config sha256:755ac014996a07c3bf534514c4b2e0f7c0b97a0985a8be3111b8d76f22d8c30c
Writing manifest to image destination
Storing signatures
FATAL: While making image from oci registry: error fetching image to cache: while building SIF from layers: conveyor failed to get: no descriptor found for reference "cf10adfeb53aa11be154fde8365c9c19854952c384b359d19fcc9d323c3b32c8"
-- Check '.nextflow.log' file for details
Is there any problem with the singularity/docker repository? It seems to me that the workflow is failing while building the SIF image....
Many thanks for your help!
Hi @greytear,
--help
command:It looks like somehow an en-dash made it into the --help
in the README (instead of two hyphens; it's –-help
but should be --help
). If you copy + paste the below, it should work
NXF_VER=23.10.1 ./nextflow run epi2me-labs/wf-bacterial-genomes --help
The README will be updated in the next release.
This could be an intermittent connectivity issue. Could you please try again and confirm that you get the same error? Thanks!
Thanks, @julibeg ! It seemed to be, indeed, an intermitent issue. Reissued the command and got it working.
Many thanks for the fabulous support!
Happy to help!
Ask away!
While trying to assist a user to run wf-bacterial-genomes on a Linux Cluster, I installed nextflow. After installation,
i tried wf-bacterial-genomes --help and got an error:
./nextflow run epi2me-labs/wf-bacterial-genomes –-help
N E X T F L O W ~ version 24.04.2
Launching
https://github.com/epi2me-labs/wf-bacterial-genomes
[focused_marconi] DSL2 - revision: 6af54574c3 [master]ERROR ~ Script compilation error
1 error
on .nextflow.log i got a mention to the same error
Jul-03 17:28:33.365 [main] DEBUG nextflow.script.ScriptRunner - Parsed script files: Jul-03 17:28:33.374 [main] ERROR nextflow.cli.Launcher - Script compilation error
1 error
Could you please provide some help on solving this? Thanks!