search

ghga-de / nf-snvcalling

a Nextflow SNV calling and annotation pipeline based on DKFZ-ODCF/SNVCallingWorkflow
MIT License
1 stars 0 forks source link

nf-core download do not download the singularity images #37

Closed pritampanda15 closed 2 months ago

pritampanda15 commented 3 months ago

Have you checked the docs?

Description of the bug

I have used nfcore-download command as suggested to pull the singularity images. But resulted in errors (see terminal outputs)

Command used and terminal output

$nf-core download ghga-de/nf-snvcalling

                                          ,--./,-.
          ___     __   __   __   ___     /,-._.--~\
    |\ | |__  __ /  ` /  \ |__) |__         }  {
    | \| |       \__, \__/ |  \ |___     \`-._,-`-,
                                          `._,._,'

    nf-core/tools version 2.14.1 - https://nf-co.re

WARNING  Could not find GitHub authentication token. Some API requests may fail.
? Select release / branch: v2.0.0  [release]
? Include the nf-core's default institutional configuration files into the download? Yes

In addition to the pipeline code, this tool can download software containers.
? Download software container images: singularity

If you are working on the same system where you will run Nextflow, you can amend the downloaded images to the ones in the$NXF_SINGULARITY_CACHEDIR folder, Nextflow will
automatically find them. However if you will transfer the downloaded files to a different system then they should be copied to the target folder.
? Copy singularity images from $NXF_SINGULARITY_CACHEDIR to the target folder or amend new images to the cache? amend

If transferring the downloaded files to another system, it can be convenient to have everything compressed in a single file.
This is not recommended when downloading Singularity images, as it can take a long time and saves very little space.
? Choose compression type: tar.gz
INFO     Saving 'ghga-de/nf-snvcalling'
          Pipeline revision: 'v2.0.0'
          Use containers: 'singularity'
          Container library: 'quay.io'
          Using $NXF_SINGULARITY_CACHEDIR': /omics/groups/OE0526/internal/users/pritam/raredisease/singularity_image'
          Output file: 'ghga-de-nf-snvcalling_v2.0.0.tar.gz'
          Include default institutional configuration: 'True'
INFO     Downloading centralised configs from GitHub
INFO     Downloading workflow files from GitHub
INFO     Processing workflow revision v2.0.0, found 8 container images in total.
Pulling singularity images ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━  62% • 5/8 completed
kubran/bcftools:1.9 WARNING: SINGULARITY_CACHEDIR and APPTAINER_CACHEDIR have different values, using the latter ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ERROR    "kubran/bcftools:1.9" failed for unclear reasons.

INFO     Pulling of "kubran/bcftools:1.9" failed.
         Please troubleshoot the command
         "singularity pull --name /omics/groups/OE0526/internal/users/pritam/raredisease/singularity_image/kubran-bcftools-1.9.img docker://quay.io/kubran/bcftools:1.9"
         manually.

ERROR    "kubran/bcftools:1.9" failed for unclear reasons.

ERROR    Pulling of "kubran/bcftools:1.9" failed.
         Please troubleshoot the command
         "singularity pull --name /omics/groups/OE0526/internal/users/pritam/raredisease/singularity_image/kubran-bcftools-1.9.img docker://quay.io/kubran/bcftools:1.9"
         manually.

ERROR    Not able to pull image of ('kubran/bcftools:1.9', '/omics/groups/OE0526/internal/users/pritam/raredisease/singularity_image/kubran-bcftools-1.9.img', None).
Pulling singularity images ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━  75% • 6/8 completed
kubran/odcf_mpileupsnvcalling:v0 WARNING: SINGULARITY_CACHEDIR and APPTAINER_CACHEDIR have different values, using the latter ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ERROR    "kubran/odcf_mpileupsnvcalling:v0" failed for unclear reasons.

INFO     Pulling of "kubran/odcf_mpileupsnvcalling:v0" failed.
         Please troubleshoot the command
         "singularity pull --name /omics/groups/OE0526/internal/users/pritam/raredisease/singularity_image/kubran-odcf_mpileupsnvcalling-v0.img
         docker://quay.io/kubran/odcf_mpileupsnvcalling:v0" manually.

ERROR    "kubran/odcf_mpileupsnvcalling:v0" failed for unclear reasons.

ERROR    Pulling of "kubran/odcf_mpileupsnvcalling:v0" failed.
         Please troubleshoot the command
         "singularity pull --name /omics/groups/OE0526/internal/users/pritam/raredisease/singularity_image/kubran-odcf_mpileupsnvcalling-v0.img
         docker://quay.io/kubran/odcf_mpileupsnvcalling:v0" manually.

ERROR    Not able to pull image of ('kubran/odcf_mpileupsnvcalling:v0',
         '/omics/groups/OE0526/internal/users/pritam/raredisease/singularity_image/kubran-odcf_mpileupsnvcalling-v0.img', None). Service might be down or internet
Pulling singularity images ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸━━━━━━━━━━━━━━━  88% • 7/8 completed
kubran/samtools:v1.9 WARNING: SINGULARITY_CACHEDIR and APPTAINER_CACHEDIR have different values, using the latter ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ERROR    "kubran/samtools:v1.9" failed for unclear reasons.

INFO     Pulling of "kubran/samtools:v1.9" failed.
         Please troubleshoot the command
         "singularity pull --name /omics/groups/OE0526/internal/users/pritam/raredisease/singularity_image/kubran-samtools-v1.9.img
         docker://quay.io/kubran/samtools:v1.9" manually.

ERROR    "kubran/samtools:v1.9" failed for unclear reasons.

ERROR    Pulling of "kubran/samtools:v1.9" failed.
         Please troubleshoot the command
         "singularity pull --name /omics/groups/OE0526/internal/users/pritam/raredisease/singularity_image/kubran-samtools-v1.9.img
         docker://quay.io/kubran/samtools:v1.9" manually.

ERROR    Not able to pull image of ('kubran/samtools:v1.9', '/omics/groups/OE0526/internal/users/pritam/raredisease/singularity_image/kubran-samtools-v1.9.img', None).
Pulling singularity images ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 8/8 completed
INFO     Compressing output into archive
INFO     Command to extract files: tar -xzf ghga-de-nf-snvcalling_v2.0.0.tar.gz
INFO     MD5 checksum for 'ghga-de-nf-snvcalling_v2.0.0.tar.gz': f5abf45510cc5c85827e4e02597d0a65

Relevant files

No response

System information

Nextflow version 23.10.1 DKFZ Cluster Singularity CenOS

kubranarci commented 3 months ago

Dear Pritam,

This pipeline is not a nf-core pipeline, thus it does not adhere to the nf-core standards completely. The mentioned image is a non-institutional. My suspect is it might be because of that. I think you can download docker images with

docker pull image

command. and then convert singularty later.

Best, Kubra

kubranarci commented 3 months ago

singularity pull --name /omics/groups/OE0526/internal/users/pritam/raredisease/singularity_image/kubran-odcf_mpileupsnvcalling-v0.img docker://quay.io/kubran/odcf_mpileupsnvcalling:v0

nf-core download command adds quay.io automatically, as I see. My docker images are in my own docker not in quay.io. since nf-core puts their docker images to quay.io, that apply to them only.

Therefore, what you can do download images one by one:

singularity pull --name /omics/groups/OE0526/internal/users/pritam/raredisease/singularity_image/kubran-odcf_mpileupsnvcalling-v0.img docker://kubran/odcf_mpileupsnvcalling:v0

pritampanda15 commented 3 months ago

Thank you @kubranarci. I will download the images manually as aforementioned.