search

nf-core / sarek

Analysis pipeline to detect germline or somatic variants (pre-processing, variant calling and annotation) from WGS / targeted sequencing
https://nf-co.re/sarek
MIT License
402 stars 404 forks source link

nf-core download sarek --outdir nf-core-sarek --container singularity --compress none --revision 3.2.1 #1092

Closed ChristianRohde closed 1 year ago

ChristianRohde commented 1 year ago

Description of the bug

Hi,

this is my first time to use not only nf-core but also nextflow. My task is to make the software running on a HPC from our university. I decided to first download the code as Singularity container. Somehow I cannot retrieve nf-core/deepvariant:1.5.0, please see below.

Best, Christian

Command used and terminal output

nf-core download sarek --outdir nf-core-sarek --container singularity --compress none --revision 3.2.1 --force

                                          ,--./,-.
          ___     __   __   __   ___     /,-._.--~\
    |\ | |__  __ /  ` /  \ |__) |__         }  {
    | \| |       \__, \__/ |  \ |___     \`-._,-`-,
                                          `._,._,'

    nf-core/tools version 2.8 - https://nf-co.re

Nextflow and nf-core can use an environment variable called $NXF_SINGULARITY_CACHEDIR that is a path to a directory where remote Singularity images are stored. This allows downloaded 
images to be cached in a central location.
? Define $NXF_SINGULARITY_CACHEDIR for a shared Singularity image download folder? [y/n]: n
WARNING  Deleting existing output directory: 'nf-core-sarek'                                                                                                                                
INFO     Saving 'nf-core/sarek'                                                                                                                                                             
          Pipeline revision: '3.2.1'                                                                                                                                                        
          Pull containers: 'singularity'                                                                                                                                                    
          Output directory: 'nf-core-sarek'                                                                                                                                                 
INFO     Downloading workflow files from GitHub                                                                                                                                             
INFO     Downloading centralised configs from GitHub                                                                                                                                        
INFO     Found 35 containers                                                                                                                                                                
Downloading singularity images ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━  46% • 16/35 completed
fgbio:2.0.2--hdfd78af_0 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 74.1% • 436.7/589.6 MB • 7.0 MB/s
gatk4:4.4.0.0--py36hdfd78af_0 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸━━━━━━━━━ 92.4% • 628.9/680.5 MB • 10.9Downloading singularity images ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━  46% • 16/35 completed
fgbio:2.0.2--hdfd78af_0 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 74.2% • 437.4/589.6 MB • 7.0 MB/s
gatk4:4.4.0.0--py36hdfd78af_0 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━ 92.6% • 629.9Downloading singularity images ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━  46% • 16/35 complet
fgbio:2.0.2--hdfd78af_0 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 74.3% • 438.1/589.6 MB • 7.0 MB
gatk4:4.4.0.0--py36hdfd78af_0 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸━━━━━━━ 92.7% • 631.0/680.5 Downloading singularity images ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━  46% • 16/35 completed
fgbio:2.0.2--hdfd78af_0 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━ 74.4% • 438.8/589.6 MB • 7.0 MB/s
gatk4:4.4.0.0--py36hdfd78af_0 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸━━━━━━ 92.9% • 632.0/680.5Downloading singularity images ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━  46% • 16/35 complete
fgbio:2.0.2--hdfd78af_0 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━ 74.5% • 439.5/589.6 MB • 7.0 MB/
gatk4:4.4.0.0--py36hdfd78af_0 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━ 93.0% • 633.1/680.5 MB Downloading singularity images ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━  46% • 16/35 completed
fgbio:2.0.2--hdfd78af_0 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 74.7% • 440.1/589.6 MB • 7.0 MB/s
gatk4:4.4.0.0--py36hdfd78af_0 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━ 93.2% • 634.1/680.5 MB • Downloading singularity images ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━  46% • 16/35 completed
fgbio:2.0.2--hdfd78af_0 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━ 74.8% • 440.9/589.6 MB • 7.0 MB/s
gatk4:4.4.0.0--py36hdfd78af_0 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸━━━━ 93.3% • 635.2/680.5 MB • 9.Downloading singularity images ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━  46% • 16/35 completed
fgbio:2.0.2--hdfd78af_0 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━ 74.9% • 441.6/589.6 MB • 7.0 MB/s
gatk4:4.4.0.0--py36hdfd78af_0 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━ 93.5% • 636.2/680.5 MB • 9.8Downloading singularity images ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━  46% • 16/35 completed
fgbio:2.0.2--hdfd78af_0 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸━━━━━━━━━━━━━━━ 75.0% • 442.3/589.6 MB • 7.0 MB/s
gatk4:4.4.0.0--py36hdfd78af_0 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸━━━ 93.6% • 637.2/680.5 MB • 10.9 MB/DDoDDownloadinPulling singularity images ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━  94% • 33/35 completed
nf-core/deepvariant:1.5.0  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━INFO     Singularity container retrieval fialed with the following error:                                                           
INFO     FATAL:   While making image from oci registry: error fetching image to cache: failed to get checksum for                   
         docker://nf-core/deepvariant:1.5.0: reading manifest 1.5.0 in docker.io/nf-core/deepvariant: errors:                       
         denied: requested access to the resource is denied                                                                         
         unauthorized: authentication required                                                                                      
Pulling singularity images ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━  94% • 33/35 completed
nf-core/deepvariant:1.5.0 unauthorized: authentication required ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
CRITICAL The container "nf-core/deepvariant:1.5.0" is unavailable.                                                                  
         FATAL:   While making image from oci registry: error fetching image to cache: failed to get checksum for                   
         docker://nf-core/deepvariant:1.5.0: reading manifest 1.5.0 in docker.io/nf-core/deepvariant: errors:                       
         denied: requested access to the resource is denied                                                                         
         unauthorized: authentication required

Relevant files

No response

System information

tdanhorn commented 1 year ago

I can confirm that. The funny thing is that I downloaded 3.2.0 just fine a couple of days ago, and I actually have a container in my cache that is labeled docker.io-google-deepvariant-1.5.0.img, so I'm not sure why it was even trying ... Has anything changed with Docker Hub?

asp8200 commented 1 year ago

I got a slightly different error msg:

nf-core download sarek --outdir nf-core-sarek --container singularity --compress none --revision 3.2.1 --force

                                          ,--./,-.
          ___     __   __   __   ___     /,-._.--~\
    |\ | |__  __ /  ` /  \ |__) |__         }  {
    | \| |       \__, \__/ |  \ |___     \`-._,-`-,
                                          `._,._,'

    nf-core/tools version 2.8 - https://nf-co.re

INFO     Saving 'nf-core/sarek'
          Pipeline revision: '3.2.1'
          Pull containers: 'singularity'
          Using $NXF_SINGULARITY_CACHEDIR': /home/ubuntu/nxf_singularity_cache
          Output directory: 'nf-core-sarek'
INFO     Downloading workflow files from GitHub
INFO     Downloading centralised configs from GitHub
INFO     Found 35 containers
Pulling singularity images ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━  94% • 33/35 completed
nf-core/deepvariant:1.5.0 Copying from cache to target directory ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
CRITICAL [Errno 2] No such file or directory: '/home/ubuntu/nxf_singularity_cache/nf-core-deepvariant-1.5.0.img'
maxulysse commented 1 year ago

@ChristianRohde @tdanhorn, thanks for letting us now the issue. I'm afraid this had to do with the recent update to the modules repo and the pipeline that switched the default custom registry from docker.io to quay.io. We're looking into it, and trying to fix nf-core/tools as soon as possible. In the meantime, you could try to run singularity pull quay.io/nf-core/deepvariant:1.5.0?

tdanhorn commented 1 year ago

I get the same message: $ singularity pull quay.io/nf-core/deepvariant:1.5.0 FATAL: While pulling library image: error fetching image: request did not succeed: UNAUTHORIZED: project quay.io not found: project quay.io not found (401 Unauthorized)

asp8200 commented 1 year ago

I get the same message: $ singularity pull quay.io/nf-core/deepvariant:1.5.0 FATAL: While pulling library image: error fetching image: request did not succeed: UNAUTHORIZED: project quay.io not found: project quay.io not found (401 Unauthorized)

Me too. Perhaps the syntax is not quite right.

asp8200 commented 1 year ago

This cmd from Maxime seems to work:

singularity pull docker://quay.io/nf-core/deepvariant:1.5.0

maxulysse commented 1 year ago

Sorry, was hard to figure the proper syntax without having Singularity on my machine, I need to reinstall it

tdanhorn commented 1 year ago

Yep, docker:// works.

ChristianRohde commented 1 year ago

Yes, I can confirm that I can download deepvariant:1.5.0 singularity image separately with the command above. However, this does not help to install sarek. I am not sure if I understand your advice correctly.

For the meantime I decided to download the previous version of sarek with the command "nf-core download sarek --outdir nf-core-sarek --container singularity --compress none --revision 3.2.0 --force". This seemed to work since it finished without any errors.

Next I run this pipeline with "nextflow run /path/2/my/nf-core-sarek/workflow/ --input /path/2/my/samplesheet.csv --outdir /path/2/my/results". Unfortunately, here the pipeline crashes at the very first step: "Workflow execution completed unsuccessfully! The exit status of the task that caused the workflow execution to fail was: 127." and finally "Command error: .command.sh: line 5: fastqc: command not found". This is unexpected since I do have a singularity image for fastqc in my singularity-images folder right within the folder where I stored the pipeline.

Where should I start troubleshooting? Unfortunately, as a beginner on this platform for me this looks a bit crowded.

Best, Christian

FriederikeHanssen commented 1 year ago

you probably need to add -profile singularityto tell nextflow to use those as far as I know there is no flag --container

FriederikeHanssen commented 1 year ago

Apologies for the issues with the download command. We are working on a fix in nf-core/tools

ChristianRohde commented 1 year ago

you probably need to add -profile singularityto tell nextflow to use those as far as I know there is no flag --container

right, it was just written below the download part in the manual: https://nf-co.re/tools/#downloading-pipelines-for-offline-use. I added this information and somehow the containers are now recognized if I interpret the tasks table from my execution_report correctly.

On the other hand now I run into another very similar but not the same error 127:
"Error executing process > 'NFCORE_SAREK:SAREK:FASTP" Command error: env: 'singularity': No such file or directory

I have no idea what could be wrong

maxulysse commented 1 year ago

Is singularity available where you're running Sarek?

tdanhorn commented 1 year ago

Hi Christian, the sarek channel on the nf-core Slack might be a better place to discuss this, since there are more people with experience, see https://nf-co.re/join if don't have an account there already. GitHub is better suited to flag specific bugs/issues with the code itself. Regarding your specific error, is there something like a module that you have to load for singularity to work (if it is installed at all)?

ChristianRohde commented 1 year ago

Hi Christian, the sarek channel on the nf-core Slack might be a better place to discuss this, since there are more people with experience, see https://nf-co.re/join if don't have an account there already. GitHub is better suited to flag specific bugs/issues with the code itself. Regarding your specific error, is there something like a module that you have to load for singularity to work (if it is installed at all)?

good points, I will join there. You spotted the problem: OMG, I forgot to load the singularity module. Now I am excited to see progress in running the pipeline

veredsil commented 1 year ago

you probably need to add -profile singularityto tell nextflow to use those as far as I know there is no flag --container

right, it was just written below the download part in the manual: https://nf-co.re/tools/#downloading-pipelines-for-offline-use. I added this information and somehow the containers are now recognized if I interpret the tasks table from my execution_report correctly.

On the other hand now I run into another very similar but not the same error 127: "Error executing process > 'NFCORE_SAREK:SAREK:FASTP" Command error: env: 'singularity': No such file or directory

I have no idea what could be wrong

Hi, did you manage to solve the error?

ChristianRohde commented 1 year ago

you probably need to add -profile singularityto tell nextflow to use those as far as I know there is no flag --container

right, it was just written below the download part in the manual: https://nf-co.re/tools/#downloading-pipelines-for-offline-use. I added this information and somehow the containers are now recognized if I interpret the tasks table from my execution_report correctly. On the other hand now I run into another very similar but not the same error 127: "Error executing process > 'NFCORE_SAREK:SAREK:FASTP" Command error: env: 'singularity': No such file or directory I have no idea what could be wrong

Hi, did you manage to solve the error?

yes, all issues resolved is the short answer. In a long answer I can add: I probably could have used sarek 3.2.1 by putting the manually downloaded image into the sarek/singularity-images/ directory. However, this was not fully clear to me as a beginner and I decided to use 3.2.0, which could be installed easily. But with later issues solved with some help on SLACK I followed this approach. The other question was a bit stupid: I forgot to load the singularity module in my script on the HPC. Once I did the pipeline was running. Now I am trying to find the configuration which works with all files available (Ensembl.GRh37 > GATK.GRCh37 > GATK.GRCh38) and how I need to format my WES regions file, but this is another story.

Best, Christian

FriederikeHanssen commented 1 year ago

With the new tools release this should be fixed now without any needed workarounds :)

FriederikeHanssen commented 1 year ago

Closing this issue as with the tools update all should work as expected. Please re-open if you encounter any more issues