SAMTOOLS_FAIDX can't locate singularity image

[PBMCHUGA]

Description of the bug

Hello,

The process SAMTOOLS_FAIDX is not able to locate the singularity image :

FATAL: could not open image /data/projects/ptdom/pipelines/rnavar/sifs/samtools-1.15.1--h1170115_0.sif : failed to retrieve path for /data/projects/ptdom/pipelines/rnavar/sifs/samtools-1.15.1--h1170115_0.sif : lstat /data/projects/ptdom/pipelines/rnavar/sifs/samtools-1.15.1--h1170115_0.sif : no such file or directory

However there is a singularity image containing samtools at this specific location available from any node on the cluster (Managed with slurm)

Image location is precised like this in the config files :

withName: 'SAMTOOLS_FAIDX' {
    container = '/data/projects/ptdom/pipelines/rnavar/sifs/samtools-1.15.1--h1170115_0.sif '
}

Command used and terminal output

No response

Relevant files

No response

System information

No response

[maxulysse]

@PBMCHUGA and you have the issue only with this process?

[PBMCHUGA]

I can't say for steps after NFCORE_RNAVAR:RNAVAR:PREPARE_GENOME:SAMTOOLS_FAIDX but at least NFCORE_RNAVAR:RNAVAR:PREPARE_GENOME:GTF2BED process is completing succesfully. You can find attached my nextflow.log file

nextflow.log

[maxulysse]

can you share what's in your cluster profile or in your custom config file?

[PBMCHUGA]

Hi,

I added

cluster {
    process {
    executor = 'slurm'
        queue = 'brouette'
        /*queue = 'servoz'*/
}

into nextflow.config.txt file.

Also it contains includeConfig 'conf/singularity.config' refering to this file

Already used this process with sarek, rnaseq and rnafusion nf-core pipelines. It worked pretty well.

[maxulysse]

I can see that you have:

    withName: 'SAMTOOLS.*' {
        container = '/data/projects/ptdom/pipelines/rnavar/sifs/samtools-1.15.1--h1170115_0.sif '
    }

In your conf/singularity.config, so I think it should cover SAMTOOL_FAIDX, can you remove the lines regarding SAMTOOL_FAIDX in your conf/singularity.config and try again?

[PBMCHUGA]

It seems it goes succesfully through the process, after singularity image pulling in the working directory specified with -work-dir

[maxulysse]

ok, I honestly have no idea why this idea fixed your problem, but I think we can close this issue then, right?

[PBMCHUGA]

Hi, It is more a workaround than a fix as we can’t have pre-built singularity image for this process. So it will re-pull singularity image at each pipeline run, unless singularity.cacheDir is properly set up. I’ll dive into it to figure out if the problem is linked to our infrastructure.

[maxulysse]

I see. I think the proper way to deal with it, is indeed to use singularity.cacheDir or NXF_SINGULARITY_CACHE_DIR. Which should allow you not to set up such custom files to your singularity files.