search

deng-lab / viroprofiler

A containerized bioinformatics pipeline for viral metagenomic data analysis
https://deng-lab.github.io/viroprofiler
MIT License
25 stars 12 forks source link

FATAL: While making image from oci registry: error fetching image to cache #13

Open Sanrrone opened 1 year ago

Sanrrone commented 1 year ago

Description of the bug

Dears, I was trying to run the setup mode but I am getting the same error about singularity even changing some flags

what could I do? thanks in advance!

Command used and terminal output

nextflow run deng-lab/viroprofiler -r main -profile singularity --mode "setup" --singularity_cachedir $new/software/singularity_cache --db $new/software/viroprofiler/db

Relevant files

N E X T F L O W  ~  version 23.04.1
Launching `https://github.com/deng-lab/viroprofiler` [cheeky_morse] DSL2 - revision: c2a1f1871b [main]

WARN: Found unexpected parameters:
* --singularity_cachedir: /scratch/project_2007362/software/singularity_cache
* --setup: true
- Ignore this warning: params.schema_ignore_params = "singularity_cachedir,setup" 

----------------------------------------------------------------------------------------------------------------
                                               __                                                                
 oooooo     oooo  o8o                        88  88                        .o88o.  o8o  oooo                     
  `888.     .8'   `"'                       88 ss 88                       888 `"  `"'  `888                     
   `888.   .8'   oooo  oooo d8b  .ooooo.     88__88    oooo d8b  .ooooo.  o888oo  oooo   888   .ooooo.  oooo d8b 
    `888. .8'    `888  `888""8P d88' `88b      ||      `888""8P d88' `88b  888    `888   888  d88' `88b `888""8P 
     `888.8'      888   888     888   888     _||_      888     888   888  888     888   888  888ooo888  888     
      `888'       888   888     888   888   // || \\    888     888   888  888     888   888  888    .o  888     
       `8'       o888o d888b    `Y8bod8P'  //      \\  d888b    `Y8bod8P' o888o   o888o o888o `Y8bod8P' d888b    
  ViroProfiler v0.2.4
----------------------------------------------------------------------------------------------------------------
Core Nextflow options
  revision       : main
  runName        : cheeky_morse
  containerEngine: singularity
  launchDir      : /scratch/project_2007362/sandro/HeP_samples/6_tax_profile/phage
  workDir        : /scratch/project_2007362/sandro/HeP_samples/6_tax_profile/phage/work
  projectDir     : /users/valensan/.nextflow/assets/deng-lab/viroprofiler
  userName       : valensan
  profile        : singularity
  configFiles    : /users/valensan/.nextflow/assets/deng-lab/viroprofiler/nextflow.config

Input/output options
  mode           : setup
  db             : /scratch/project_2007362/software/viroprofiler/db
  outdir         : output

QC
  contamref_idx  : /users/valensan/viroprofiler/contamination_refs/hg19/ref

Contig library parameters
  assemblies     : scaffolds

Others
  use_iphop      : true
  use_dram       : true

Max job request options
  max_cpus       : 1
  max_memory     : 8.GB
  max_time       : 12.h

!! Only displaying parameters that differ from the pipeline defaults !!
----------------------------------------------------------------------------------------------------------------
If you use ViroProfiler for your analysis please cite:

* The ViroProfiler pipeline
 Ru, Jinlong, et al. "ViroProfiler: a containerized bioinformatics pipeline for viral metagenomic data analysis."
 Gut Microbes 15.1 (2023): 2192522. https://doi.org/10.1080/19490976.2023.2192522

* The nf-core framework
 Ewels, Philip A., et al. "The nf-core framework for community-curated bioinformatics pipelines."
 Nature biotechnology 38.3 (2020): 276-278. https://doi.org/10.1038/s41587-020-0439-x

* Software dependencies
  https://github.com/deng-lab/viroprofiler/blob/main/CITATIONS.md
----------------------------------------------------------------------------------------------------------------
[-        ] process > VIROPROFILER:SETUP:DB_CHECKV   -
[-        ] process > VIROPROFILER:SETUP:DB_VIBRANT  -
[-        ] process > VIROPROFILER:SETUP:DB_VIRSO... -
[-        ] process > VIROPROFILER:SETUP:DB_DRAM     -
[-        ] process > VIROPROFILER:SETUP:DB_IPHOP    -
[-        ] process > VIROPROFILER:SETUP:DB_VREFSEQ  -
Pulling Singularity image docker://denglab/viroprofiler-virsorter2:v0.2.5 [cache /scratch/project_2007362/software/singularity/cache/denglab-viroprofiler-virsorter2-v0.2.5.img]
Pulling Singularity image docker://denglab/viroprofiler-geneannot:v0.2 [cache /scratch/project_2007362/software/singularity/cache/denglab-viroprofiler-geneannot-v0.2.img]
Pulling Singularity image docker://denglab/viroprofiler-base:v0.2 [cache /scratch/project_2007362/software/singularity/cache/denglab-viroprofiler-base-v0.2.img]
Pulling Singularity image docker://denglab/viroprofiler-vibrant:v0.2 [cache /scratch/project_2007362/software/singularity/cache/denglab-viroprofiler-vibrant-v0.2.img]
Pulling Singularity image docker://denglab/viroprofiler-host:v0.1 [cache /scratch/project_2007362/software/singularity/cache/denglab-viroprofiler-host-v0.1.img]

[-        ] process > VIROPROFILER:SETUP:DB_CHECKV   -
[-        ] process > VIROPROFILER:SETUP:DB_VIBRANT  -
[-        ] process > VIROPROFILER:SETUP:DB_VIRSO... -
[-        ] process > VIROPROFILER:SETUP:DB_DRAM     -
[-        ] process > VIROPROFILER:SETUP:DB_IPHOP    -
[-        ] process > VIROPROFILER:SETUP:DB_VREFSEQ  -
Pulling Singularity image docker://denglab/viroprofiler-virsorter2:v0.2.5 [cache /scratch/project_2007362/software/singularity/cache/denglab-viroprofiler-virsorter2-v0.2.5.img]
Pulling Singularity image docker://denglab/viroprofiler-geneannot:v0.2 [cache /scratch/project_2007362/software/singularity/cache/denglab-viroprofiler-geneannot-v0.2.img]
Pulling Singularity image docker://denglab/viroprofiler-base:v0.2 [cache /scratch/project_2007362/software/singularity/cache/denglab-viroprofiler-base-v0.2.img]
Pulling Singularity image docker://denglab/viroprofiler-vibrant:v0.2 [cache /scratch/project_2007362/software/singularity/cache/denglab-viroprofiler-vibrant-v0.2.img]
Pulling Singularity image docker://denglab/viroprofiler-host:v0.1 [cache /scratch/project_2007362/software/singularity/cache/denglab-viroprofiler-host-v0.1.img]
ERROR ~ Error executing process > 'VIROPROFILER:SETUP:DB_VIBRANT'

Caused by:
  Failed to pull singularity image
  command: singularity pull  --name denglab-viroprofiler-vibrant-v0.2.img.pulling.1689064673556 docker://denglab/viroprofiler-vibrant:v0.2 > /dev/null
  status : 255
  message:
    INFO:    Converting OCI blobs to SIF format
    WARNING: 'nodev' mount option set on /scratch, it could be a source of failure during build process
    INFO:    Starting build...
    Getting image source signatures
    Copying blob sha256:7a6db449b51b92eac5c81cdbd82917785343f1664b2be57b22337b0a40c5b29d
    Copying blob sha256:b51904ca3834848d576e609bab4b382028643e662db724a58edb9401f9af220c
    Copying blob sha256:16f76cf3882a90e2e7969b03b7d0147c189cfb3966d14800091466f2d5fc0a51
    Copying blob sha256:377cc88d7c7e949460cc24d07979f2e428f2482e0b360d72860ff1df404208e0
    Copying blob sha256:7a15ae76cb488a6adde55d23dfd5571c57bf02206168b7795f7b473b1dc7c218
    Copying blob sha256:d0e88a31bdb313b5fab1a325bc79ad5c487cb722dbf65b1561fd4810ebd7b717
    Copying blob sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1
    Copying blob sha256:301aa5df7f62dcf710fc30e41d585887f918992bc71dba116c7edc267a3c3422
    Copying blob sha256:26142b6b2c1ea5de9b7566a9862ea2e9beb02b1a059295cd221ec89c75aac1e8
    Copying blob sha256:a7c07239fd155305c23fdbe3cb67838b8d57e0d5922d01b643ec5da4dbf7c7d7
    Copying blob sha256:1a9f62d1ab924f868aef3cbe75eacfe9f255289fbc104d1b592d830f07e6b0bd
    Copying blob sha256:43efb7cfae927b3a5717e3b6e6ed2bae4e0cac21ecb32f0086f699d6ecc89dcc
    Copying blob sha256:01e63aa1ed13079d11b79bbde5ebf2e2c0499d4274ad5cbe3a6af5cd6f4ef361
    Copying config sha256:aff1e9de36d4fbec03924d87c9c2ed625134b923d6c427a04740314f05be3f10
    Writing manifest to image destination
    Storing signatures
    FATAL:   While making image from oci registry: error fetching image to cache: while building SIF from layers: conveyor failed to get: no descriptor found for reference "70ec07d437eae4004a69d5c7d74c25d6d3384caf5e215753eed1e402653a3ab6"

 -- Check '.nextflow.log' file for details
-[ViroProfiler] Pipeline completed with errors-

[-        ] process > VIROPROFILER:SETUP:DB_CHECKV   -
[-        ] process > VIROPROFILER:SETUP:DB_VIBRANT  -
[-        ] process > VIROPROFILER:SETUP:DB_VIRSO... -
[-        ] process > VIROPROFILER:SETUP:DB_DRAM     -
[-        ] process > VIROPROFILER:SETUP:DB_IPHOP    -
[-        ] process > VIROPROFILER:SETUP:DB_VREFSEQ  -
Pulling Singularity image docker://denglab/viroprofiler-virsorter2:v0.2.5 [cache /scratch/project_2007362/software/singularity/cache/denglab-viroprofiler-virsorter2-v0.2.5.img]
Pulling Singularity image docker://denglab/viroprofiler-geneannot:v0.2 [cache /scratch/project_2007362/software/singularity/cache/denglab-viroprofiler-geneannot-v0.2.img]
Pulling Singularity image docker://denglab/viroprofiler-base:v0.2 [cache /scratch/project_2007362/software/singularity/cache/denglab-viroprofiler-base-v0.2.img]
Pulling Singularity image docker://denglab/viroprofiler-vibrant:v0.2 [cache /scratch/project_2007362/software/singularity/cache/denglab-viroprofiler-vibrant-v0.2.img]
Pulling Singularity image docker://denglab/viroprofiler-host:v0.1 [cache /scratch/project_2007362/software/singularity/cache/denglab-viroprofiler-host-v0.1.img]
ERROR ~ Error executing process > 'VIROPROFILER:SETUP:DB_VIBRANT'

Caused by:
  Failed to pull singularity image
  command: singularity pull  --name denglab-viroprofiler-vibrant-v0.2.img.pulling.1689064673556 docker://denglab/viroprofiler-vibrant:v0.2 > /dev/null
  status : 255
  message:
    INFO:    Converting OCI blobs to SIF format
    WARNING: 'nodev' mount option set on /scratch, it could be a source of failure during build process
    INFO:    Starting build...
    Getting image source signatures
    Copying blob sha256:7a6db449b51b92eac5c81cdbd82917785343f1664b2be57b22337b0a40c5b29d
    Copying blob sha256:b51904ca3834848d576e609bab4b382028643e662db724a58edb9401f9af220c
    Copying blob sha256:16f76cf3882a90e2e7969b03b7d0147c189cfb3966d14800091466f2d5fc0a51
    Copying blob sha256:377cc88d7c7e949460cc24d07979f2e428f2482e0b360d72860ff1df404208e0
    Copying blob sha256:7a15ae76cb488a6adde55d23dfd5571c57bf02206168b7795f7b473b1dc7c218
    Copying blob sha256:d0e88a31bdb313b5fab1a325bc79ad5c487cb722dbf65b1561fd4810ebd7b717
    Copying blob sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1
    Copying blob sha256:301aa5df7f62dcf710fc30e41d585887f918992bc71dba116c7edc267a3c3422
    Copying blob sha256:26142b6b2c1ea5de9b7566a9862ea2e9beb02b1a059295cd221ec89c75aac1e8
    Copying blob sha256:a7c07239fd155305c23fdbe3cb67838b8d57e0d5922d01b643ec5da4dbf7c7d7
    Copying blob sha256:1a9f62d1ab924f868aef3cbe75eacfe9f255289fbc104d1b592d830f07e6b0bd
    Copying blob sha256:43efb7cfae927b3a5717e3b6e6ed2bae4e0cac21ecb32f0086f699d6ecc89dcc
    Copying blob sha256:01e63aa1ed13079d11b79bbde5ebf2e2c0499d4274ad5cbe3a6af5cd6f4ef361
    Copying config sha256:aff1e9de36d4fbec03924d87c9c2ed625134b923d6c427a04740314f05be3f10
    Writing manifest to image destination
    Storing signatures
    FATAL:   While making image from oci registry: error fetching image to cache: while building SIF from layers: conveyor failed to get: no descriptor found for reference "70ec07d437eae4004a69d5c7d74c25d6d3384caf5e215753eed1e402653a3ab6"

System information

nextflow: v23.04.1 Hardware: HPC Executor: slurm Container: singularity OS: RHEL7 Version of viroprofiler: main (also same result with 0.2.5)

mhmism commented 1 year ago

Same issue here. You can bypass this error by manually pulling the singularity dockers. However, following this step, I am faced with the issue that the database directory is read-only and is not writable, and I cannot proceed further.

zachflam commented 1 year ago

Same issue.

System information: nextflow: v23.04.1 Hardware: HPC Executor: slurm Container: singularity OS: RHEL8.7 Version of viroprofiler: v0.2.4