singularity.registry ignored when pulling nf-core/bedtools_coreutils image

mathob commented 1 day ago

Description of the bug

From terminal output (see below) I see that nextflow tries unsuccessfully to run this:

singularity pull --name nf-core-bedtools_coreutils-a623c13f66d5262b.img.pulling.1732158845794 docker://nf-core/bedtools_coreutils:a623c13f66d5262b > /dev/null

When I try repeating this command without redirection I get this:

singularity pull  --name nf-core-bedtools_coreutils-a623c13f66d5262b.img.pulling.1732158845794 docker://nf-core/bedtools_coreutils:a623c13f66d5262b
WARNING: Couldn't use cached digest for registry: HEAD https://index.docker.io/v2/nf-core/bedtools_coreutils/manifests/a623c13f66d5262b: unexpected status code 401 Unauthorized (HEAD responses have no body, use GET for details)
WARNING: Falling back to direct digest.
FATAL:   While making image from oci registry: error fetching image to cache: failed to get checksum for docker://nf-core/bedtools_coreutils:a623c13f66d5262b: GET https://index.docker.io/v2/nf-core/bedtools_coreutils/manifests/a623c13f66d5262b: UNAUTHORIZED: authentication required; [map[Action:pull Class: Name:nf-core/bedtools_coreutils Type:repository]]

Command used and terminal output

nextflow run nf-core/fetchngs --input ./2ids.tsv --outdir ./results --nf_core_pipeline rnaseq -profile pawsey_setonix -c copy_queue.config

excerpt of output:

Pulling Singularity image https://depot.galaxyproject.org/singularity/stringtie:2.2.3--h43eeafb_0 [cache /software/projects/pawsey1119/mhobbs/.nextflow_singularity/depot.galaxyproject.org-singularity-stringtie-2.2.3--h43eeafb_0.img]
Pulling Singularity image https://depot.galaxyproject.org/singularity/qualimap:2.3--hdfd78af_0 [cache /software/projects/pawsey1119/mhobbs/.nextflow_singularity/depot.galaxyproject.org-singularity-qualimap-2.3--hdfd78af_0.img]      Pulling Singularity image https://depot.galaxyproject.org/singularity/subread:2.0.6--he4a0461_2 [cache /software/projects/pawsey1119/mhobbs/.nextflow_singularity/depot.galaxyproject.org-singularity-subread-2.0.6--he4a0461_2.img]
Pulling Singularity image docker://nf-core/bedtools_coreutils:a623c13f66d5262b [cache /software/projects/pawsey1119/mhobbs/.nextflow_singularity/nf-core-bedtools_coreutils-a623c13f66d5262b.img]
Pulling Singularity image https://depot.galaxyproject.org/singularity/bioconductor-dupradar:1.32.0--r43hdfd78af_0 [cache /software/projects/pawsey1119/mhobbs/.nextflow_singularity/depot.galaxyproject.org-singularity-bioconductor-dupradar-1.32.0--r43hdfd78af_0.img]
ERROR ~ Error executing process > 'NFCORE_RNASEQ:RNASEQ:BEDTOOLS_GENOMECOV_FW (1)'
   Caused by:                                         Failed to pull singularity image
    command: singularity pull  --name nf-core-bedtools_coreutils-a623c13f66d5262b.img.pulling.1732158845794 docker://nf-core/bedtools_coreutils:a623c13f66d5262b > /dev/null
    status : 255
    hint   : Try and increase singularity.pullTimeout in the config (current is "20m")
    message:
      WARNING: Couldn't use cached digest for registry: HEAD https://index.docker.io/v2/nf-core/bedtools_coreutils/manifests/a623c13f66d5262b: unexpected status code 401 Unauthorized (HEAD responses have no body, use GET for details)
      WARNING: Falling back to direct digest.
      FATAL:   While making image from oci registry: error fetching image to cache: failed to get checksum for docker://nf-core/bedtools_coreutils:a623c13f66d5262b: GET https://index.docker.io/v2/nf-core/bedtools_coreutils/manifests/a623c13f66d5262b: UNAUTHORIZED: authentication required; [map[Action:pull Class: Name:nf-core/bedtools_coreutils Type:repository]]

 -- Check '.nextflow.log' file for details
ERROR ~ Pipeline failed. Please refer to troubleshooting docs: https://nf-co.re/docs/usage/troubleshooting

 -- Check '.nextflow.log' file for details
-[nf-core/rnaseq] Pipeline completed with errors-

Relevant files

No response

System information

nf-core/rnaseq: v3.17.0-g00f924c
Nextflow: 24.04.3

mathob commented 1 day ago

Also wondering about whether it should be "docker://nfcore/" rather than "docker://nf-core/"?

mathob commented 1 day ago

ahh, I pasted in the wrong command, sorry, it was

nextflow run nf-core/rnaseq \ --input results/samplesheet/samplesheet.csv \ --trimmer fastp --extra_fastp_args '--reads_to_process 10000' \ --outdir rnaseq_results_10kreads \ --gtf gencode.v47.annotation.gtf.gz \ --gencode true \ --fasta GRCh38.p14.genome.fa.gz \ --additional_fasta ERCC92.fa \ --star_index false \ --save_reference true \ --gene_bed false \ --aligner star_rsem \ -profile pawsey_setonix,singularity

The output including error messages is correct though

mathob commented 1 day ago

In another system I successfully ran a previous version of the pipeline with the same inputs

nf-core/rnaseq: v3.16.0-g33df0c0
Nextflow: 24.04.1

This run retreived an alternative image (community-cr-prod.seqera.io-docker-registry-v2-blobs-sha256-63-6397750e9730a3fbcc5b4c43f14bd141c64c723fd7dad80e47921a68a7c3cd2 1-data.img) - this alternative was removed from modules/nf-core/bedtools/genomecov/main.nf in https://github.com/nf-core/rnaseq/commit/14e76774942a3fe4c9837541af5edb8457cc640c#diff-2c1c983c31b62b0f399bb6f80772337aa71b9c4951692835c050e6b4f90e9c30L8

mathob commented 1 day ago

After more reading I see that since the config does this:

singularity.registry = 'quay.io'

then the URI used in the singularity pull command should be changed to "docker://quay.io/nf-core/bedtools_coreutils:a623c13f66d5262b" - but isn't

nf-core / rnaseq