Network unreachable nodes

[nmendozam]

Description of the bug

When running the pipeline in a slurm HPC environment, the nodes might not have access to the network. Here it seems that the SAMTOOLS_STATS process is trying to access the network.

ERROR ~ Error executing process > 'NFCORE_BAMTOFASTQ:BAMTOFASTQ:PRE_CONVERSION_QC:SAMTOOLS_STATS (KIEL_109360_AGE00233)'

Caused by:
  Process `NFCORE_BAMTOFASTQ:BAMTOFASTQ:PRE_CONVERSION_QC:SAMTOOLS_STATS (KIEL_109360_AGE00233)` terminated with an error exit status (254)

Command executed:

  samtools \
      stats \
      --threads 1 \
       \
      smaple.cram \
      > sample.stats

  cat <<-END_VERSIONS > versions.yml
  "NFCORE_BAMTOFASTQ:BAMTOFASTQ:PRE_CONVERSION_QC:SAMTOOLS_STATS":
      samtools: $(echo $(samtools --version 2>&1) | sed 's/^.*samtools //; s/Using.*$//')
  END_VERSIONS

Command exit status:
  254

Command output:
  (empty)

Command error:
  WARNING: destination /work_beegfs/sukmb618 already in mount list: destination is already in the mount point list
  [W::find_file_url] Failed to open reference "https://www.ebi.ac.uk/ena/cram/md5/6aef897c3d6ff0c78aff06ac189178dd": Network is unreachable
  [E::fai_build3_core] Failed to open the file /home/dnanexus/genome.fa
  [E::refs_load_fai] Failed to open reference file '/home/dnanexus/genome.fa'
  [W::cram_get_ref] Failed to populate reference for id 0
  [E::cram_decode_slice] Unable to fetch reference #0:10029-187793

  [W::find_file_url] Failed to open reference "https://www.ebi.ac.uk/ena/cram/md5/6aef897c3d6ff0c78aff06ac189178dd": Network is unreachable
  [E::cram_next_slice] Slice decode failure
  Failure while decoding file
  [E::fai_build3_core] Failed to open the file /home/dnanexus/genome.fa
  [E::refs_load_fai] Failed to open reference file '/home/dnanexus/genome.fa'
  [W::cram_get_ref] Failed to populate reference for id 0
  [E::cram_decode_slice] Unable to fetch reference #0:187720-930362

Is there a way to make this processes fail gracefully by ignoring these errors? Because as long as I can tell, this process is cd for statistics.

Command used and terminal output

nextflow run nf-core/bamtofastq \
   -profile singularity \
   --input samplesheet.csv \
   --outdir outdir

Relevant files

No response

System information

• Nextflow 23.04.0
• Hardware HPC
• Executor slurm
• Container engine Singularity

[nmendozam]

The issue was that I did not specify the reference file manually for the .cram files as stated here. The instructions on how to specify the reference manually are available in docs/usage.md which was not clear to me at first sight.

https://github.com/nf-core/bamtofastq/blob/c70f49b20c83c72a0e6198e4b13edb5e2fb14725/docs/usage.md?plain=1#L153-L156

[FriederikeHanssen]

Great it was solved. Is there a better place you would expect this information to be?

[min-codes]

Great it was solved. Is there a better place you would expect this information to be?

Not the author, but I would expect this to be on the samtools-fastq documentation page