search

nf-core / circdna

Pipeline for the identification of extrachromosomal circular DNA (ecDNA) from Circle-seq, WGS, and ATAC-seq data that were generated from cancer and other eukaryotic cells.
https://nf-co.re/circdna
MIT License
25 stars 14 forks source link

genome.fa: No such file or directory #45

Closed ShixiangWang closed 1 year ago

ShixiangWang commented 1 year ago

Description of the bug

It's strange to observe that commands to run samtools sort report errors related to a fasta file with "cram" labelled.

Command used and terminal output

[0c/beb229] process > NFCORE_CIRCDNA:CIRCDNA:INPUT_CHECK:SAMPLESHEET_CHECK (samplesheet_snu16_circleseq... [100%] 1 of 1 ✔
[-        ] process > NFCORE_CIRCDNA:CIRCDNA:CAT_FASTQ                                                     -
[80/3c44bc] process > NFCORE_CIRCDNA:CIRCDNA:FASTQC (SNU16-HMW)                                            [100%] 2 of 2, cached: 2 ✔
[19/813d2c] process > NFCORE_CIRCDNA:CIRCDNA:TRIMGALORE (SNU16-ZHI)                                        [100%] 2 of 2, cached: 2 ✔
[96/00ac24] process > NFCORE_CIRCDNA:CIRCDNA:BWA_INDEX (genome.fa)                                         [100%] 1 of 1, cached: 1 ✔
[7c/032898] process > NFCORE_CIRCDNA:CIRCDNA:BWA_MEM (SNU16-HMW)                                           [100%] 2 of 2, cached: 2 ✔
[c9/30c7d4] process > NFCORE_CIRCDNA:CIRCDNA:SAMTOOLS_INDEX_BAM (SNU16-HMW)                                [100%] 2 of 2, cached: 2 ✔
[71/4b7aaa] process > NFCORE_CIRCDNA:CIRCDNA:BAM_STATS_SAMTOOLS_RAW:SAMTOOLS_STATS (SNU16-HMW)             [100%] 2 of 2, cached: 2 ✔
[45/8b4bc4] process > NFCORE_CIRCDNA:CIRCDNA:BAM_STATS_SAMTOOLS_RAW:SAMTOOLS_FLAGSTAT (SNU16-HMW)          [100%] 2 of 2, cached: 2 ✔
[e1/ef10d2] process > NFCORE_CIRCDNA:CIRCDNA:BAM_STATS_SAMTOOLS_RAW:SAMTOOLS_IDXSTATS (SNU16-HMW)          [100%] 2 of 2, cached: 2 ✔
[48/4a0270] process > NFCORE_CIRCDNA:CIRCDNA:MARK_DUPLICATES_PICARD:PICARD_MARKDUPLICATES (SNU16-HMW)      [100%] 2 of 2, cached: 2 ✔
[77/be7b27] process > NFCORE_CIRCDNA:CIRCDNA:MARK_DUPLICATES_PICARD:SAMTOOLS_INDEX (SNU16-HMW)             [100%] 2 of 2, cached: 2 ✔
[93/c0e657] process > NFCORE_CIRCDNA:CIRCDNA:MARK_DUPLICATES_PICARD:BAM_STATS_SAMTOOLS:SAMTOOLS_STATS (... [100%] 2 of 2, cached: 2 ✔
[5f/b87a29] process > NFCORE_CIRCDNA:CIRCDNA:MARK_DUPLICATES_PICARD:BAM_STATS_SAMTOOLS:SAMTOOLS_FLAGSTA... [100%] 2 of 2, cached: 2 ✔
[29/8e65bd] process > NFCORE_CIRCDNA:CIRCDNA:MARK_DUPLICATES_PICARD:BAM_STATS_SAMTOOLS:SAMTOOLS_IDXSTAT... [100%] 2 of 2, cached: 2 ✔
[2e/b22b8c] process > NFCORE_CIRCDNA:CIRCDNA:SAMTOOLS_VIEW_FILTER (SNU16-HMW)                              [100%] 2 of 2, cached: 2 ✔
[c8/a1b395] process > NFCORE_CIRCDNA:CIRCDNA:SAMTOOLS_SORT_FILTERED (SNU16-HMW)                            [100%] 2 of 2, failed: 2 ✘
[-        ] process > NFCORE_CIRCDNA:CIRCDNA:SAMTOOLS_INDEX_FILTERED                                       -
[-        ] process > NFCORE_CIRCDNA:CIRCDNA:SAMTOOLS_SORT_QNAME_CF                                        -
[-        ] process > NFCORE_CIRCDNA:CIRCDNA:SAMBLASTER                                                    -
[-        ] process > NFCORE_CIRCDNA:CIRCDNA:BEDTOOLS_SPLITBAM2BED                                         -
[-        ] process > NFCORE_CIRCDNA:CIRCDNA:BEDTOOLS_SORTEDBAM2BED                                        -
[-        ] process > NFCORE_CIRCDNA:CIRCDNA:CIRCLEFINDER                                                  -
[-        ] process > NFCORE_CIRCDNA:CIRCDNA:SAMTOOLS_SORT_QNAME_CM                                        -
[-        ] process > NFCORE_CIRCDNA:CIRCDNA:CIRCLEMAP_READEXTRACTOR                                       -
[-        ] process > NFCORE_CIRCDNA:CIRCDNA:SAMTOOLS_SORT_RE                                              -
[-        ] process > NFCORE_CIRCDNA:CIRCDNA:SAMTOOLS_INDEX_RE                                             -
[-        ] process > NFCORE_CIRCDNA:CIRCDNA:CIRCLEMAP_REPEATS                                             -
[-        ] process > NFCORE_CIRCDNA:CIRCDNA:CIRCLEMAP_REALIGN                                             -
[-        ] process > NFCORE_CIRCDNA:CIRCDNA:CIRCEXPLORER2_PARSE                                           -
[-        ] process > NFCORE_CIRCDNA:CIRCDNA:CUSTOM_DUMPSOFTWAREVERSIONS                                   -
[-        ] process > NFCORE_CIRCDNA:CIRCDNA:MULTIQC                                                       -
-[nf-core/circdna] Pipeline completed with errors-

Error executing process > 'NFCORE_CIRCDNA:CIRCDNA:SAMTOOLS_SORT_FILTERED (SNU16-ZHI)'

Caused by:
  Process `NFCORE_CIRCDNA:CIRCDNA:SAMTOOLS_SORT_FILTERED (SNU16-ZHI)` terminated with an error exit status (1)

Command executed:

  samtools sort  -@ 12 -o SNU16-ZHI.md.filtered.sorted.bam -T SNU16-ZHI.md.filtered.sorted SNU16-ZHI.md.filtered.bam
  cat <<-END_VERSIONS > versions.yml
  "NFCORE_CIRCDNA:CIRCDNA:SAMTOOLS_SORT_FILTERED":
      samtools: $(echo $(samtools --version 2>&1) | sed 's/^.*samtools //; s/Using.*$//')
  END_VERSIONS

Command exit status:
  1

Command output:
  (empty)

Command error:
  /data3/wsx/nxf/work/29/2381d613e63ff7f2e52f9eabcba45e/genome.fa: No such file or directory
  [E::refs_load_fai] Failed to open reference file '/data3/wsx/nxf/work/29/2381d613e63ff7f2e52f9eabcba45e/genome.fa'
  [E::cram_get_ref] Failed to populate reference for id 0
  [E::cram_decode_slice] Unable to fetch reference #0:1412516-1418583

  /data3/wsx/nxf/work/29/2381d613e63ff7f2e52f9eabcba45e/genome.fa: No such file or directory
  [E::refs_load_fai] Failed to open reference file '/data3/wsx/nxf/work/29/2381d613e63ff7f2e52f9eabcba45e/genome.fa'
  [E::cram_get_ref] Failed to populate reference for id 0
  [E::cram_decode_slice] Unable to fetch reference #0:1418434-1418633

  /data3/wsx/nxf/work/29/2381d613e63ff7f2e52f9eabcba45e/genome.fa: No such file or directory
  [E::refs_load_fai] Failed to open reference file '/data3/wsx/nxf/work/29/2381d613e63ff7f2e52f9eabcba45e/genome.fa'
  [E::cram_get_ref] Failed to populate reference for id 0
  [E::cram_decode_slice] Unable to fetch reference #0:1418484-1418709

  /data3/wsx/nxf/work/29/2381d613e63ff7f2e52f9eabcba45e/genome.fa: No such file or directory
  [E::refs_load_fai] Failed to open reference file '/data3/wsx/nxf/work/29/2381d613e63ff7f2e52f9eabcba45e/genome.fa'
  [E::cram_get_ref] Failed to populate reference for id 0
  [E::cram_decode_slice] Unable to fetch reference #0:1418560-1418792

  /data3/wsx/nxf/work/29/2381d613e63ff7f2e52f9eabcba45e/genome.fa: No such file or directory
  [E::refs_load_fai] Failed to open reference file '/data3/wsx/nxf/work/29/2381d613e63ff7f2e52f9eabcba45e/genome.fa'
executor >  local (3)
[0c/beb229] process > NFCORE_CIRCDNA:CIRCDNA:INPUT_CHECK:SAMPLESHEET_CHECK (samplesheet_snu16_circleseq... [100%] 1 of 1 ✔
[-        ] process > NFCORE_CIRCDNA:CIRCDNA:CAT_FASTQ                                                     -
[80/3c44bc] process > NFCORE_CIRCDNA:CIRCDNA:FASTQC (SNU16-HMW)                                            [100%] 2 of 2, cached: 2 ✔
[19/813d2c] process > NFCORE_CIRCDNA:CIRCDNA:TRIMGALORE (SNU16-ZHI)                                        [100%] 2 of 2, cached: 2 ✔
[96/00ac24] process > NFCORE_CIRCDNA:CIRCDNA:BWA_INDEX (genome.fa)                                         [100%] 1 of 1, cached: 1 ✔
[7c/032898] process > NFCORE_CIRCDNA:CIRCDNA:BWA_MEM (SNU16-HMW)                                           [100%] 2 of 2, cached: 2 ✔
[c9/30c7d4] process > NFCORE_CIRCDNA:CIRCDNA:SAMTOOLS_INDEX_BAM (SNU16-HMW)                                [100%] 2 of 2, cached: 2 ✔
[71/4b7aaa] process > NFCORE_CIRCDNA:CIRCDNA:BAM_STATS_SAMTOOLS_RAW:SAMTOOLS_STATS (SNU16-HMW)             [100%] 2 of 2, cached: 2 ✔
[45/8b4bc4] process > NFCORE_CIRCDNA:CIRCDNA:BAM_STATS_SAMTOOLS_RAW:SAMTOOLS_FLAGSTAT (SNU16-HMW)          [100%] 2 of 2, cached: 2 ✔
[e1/ef10d2] process > NFCORE_CIRCDNA:CIRCDNA:BAM_STATS_SAMTOOLS_RAW:SAMTOOLS_IDXSTATS (SNU16-HMW)          [100%] 2 of 2, cached: 2 ✔
[48/4a0270] process > NFCORE_CIRCDNA:CIRCDNA:MARK_DUPLICATES_PICARD:PICARD_MARKDUPLICATES (SNU16-HMW)      [100%] 2 of 2, cached: 2 ✔
[77/be7b27] process > NFCORE_CIRCDNA:CIRCDNA:MARK_DUPLICATES_PICARD:SAMTOOLS_INDEX (SNU16-HMW)             [100%] 2 of 2, cached: 2 ✔
[93/c0e657] process > NFCORE_CIRCDNA:CIRCDNA:MARK_DUPLICATES_PICARD:BAM_STATS_SAMTOOLS:SAMTOOLS_STATS (... [100%] 2 of 2, cached: 2 ✔
[5f/b87a29] process > NFCORE_CIRCDNA:CIRCDNA:MARK_DUPLICATES_PICARD:BAM_STATS_SAMTOOLS:SAMTOOLS_FLAGSTA... [100%] 2 of 2, cached: 2 ✔
[29/8e65bd] process > NFCORE_CIRCDNA:CIRCDNA:MARK_DUPLICATES_PICARD:BAM_STATS_SAMTOOLS:SAMTOOLS_IDXSTAT... [100%] 2 of 2, cached: 2 ✔
[2e/b22b8c] process > NFCORE_CIRCDNA:CIRCDNA:SAMTOOLS_VIEW_FILTER (SNU16-HMW)                              [100%] 2 of 2, cached: 2 ✔
[c8/a1b395] process > NFCORE_CIRCDNA:CIRCDNA:SAMTOOLS_SORT_FILTERED (SNU16-HMW)                            [100%] 2 of 2, failed: 2 ✘
[-        ] process > NFCORE_CIRCDNA:CIRCDNA:SAMTOOLS_INDEX_FILTERED                                       -
[-        ] process > NFCORE_CIRCDNA:CIRCDNA:SAMTOOLS_SORT_QNAME_CF                                        -
[-        ] process > NFCORE_CIRCDNA:CIRCDNA:SAMBLASTER                                                    -
[-        ] process > NFCORE_CIRCDNA:CIRCDNA:BEDTOOLS_SPLITBAM2BED                                         -
[-        ] process > NFCORE_CIRCDNA:CIRCDNA:BEDTOOLS_SORTEDBAM2BED                                        -
[-        ] process > NFCORE_CIRCDNA:CIRCDNA:CIRCLEFINDER                                                  -
[-        ] process > NFCORE_CIRCDNA:CIRCDNA:SAMTOOLS_SORT_QNAME_CM                                        -
[-        ] process > NFCORE_CIRCDNA:CIRCDNA:CIRCLEMAP_READEXTRACTOR                                       -
[-        ] process > NFCORE_CIRCDNA:CIRCDNA:SAMTOOLS_SORT_RE                                              -
[-        ] process > NFCORE_CIRCDNA:CIRCDNA:SAMTOOLS_INDEX_RE                                             -
[-        ] process > NFCORE_CIRCDNA:CIRCDNA:CIRCLEMAP_REPEATS                                             -
[-        ] process > NFCORE_CIRCDNA:CIRCDNA:CIRCLEMAP_REALIGN                                             -
[-        ] process > NFCORE_CIRCDNA:CIRCDNA:CIRCEXPLORER2_PARSE                                           -
[-        ] process > NFCORE_CIRCDNA:CIRCDNA:CUSTOM_DUMPSOFTWAREVERSIONS                                   -
[-        ] process > NFCORE_CIRCDNA:CIRCDNA:MULTIQC                                                       -
-[nf-core/circdna] Pipeline completed with errors-
Error executing process > 'NFCORE_CIRCDNA:CIRCDNA:SAMTOOLS_SORT_FILTERED (SNU16-ZHI)'

Caused by:
  Process `NFCORE_CIRCDNA:CIRCDNA:SAMTOOLS_SORT_FILTERED (SNU16-ZHI)` terminated with an error exit status (1)

Command executed:

  samtools sort  -@ 12 -o SNU16-ZHI.md.filtered.sorted.bam -T SNU16-ZHI.md.filtered.sorted SNU16-ZHI.md.filtered.bam
  cat <<-END_VERSIONS > versions.yml
  "NFCORE_CIRCDNA:CIRCDNA:SAMTOOLS_SORT_FILTERED":
      samtools: $(echo $(samtools --version 2>&1) | sed 's/^.*samtools //; s/Using.*$//')
  END_VERSIONS

Command exit status:
  1

Command output:
  (empty)

Command error:
  /data3/wsx/nxf/work/29/2381d613e63ff7f2e52f9eabcba45e/genome.fa: No such file or directory
  [E::refs_load_fai] Failed to open reference file '/data3/wsx/nxf/work/29/2381d613e63ff7f2e52f9eabcba45e/genome.fa'
  [E::cram_get_ref] Failed to populate reference for id 0
  [E::cram_decode_slice] Unable to fetch reference #0:1412516-1418583

  /data3/wsx/nxf/work/29/2381d613e63ff7f2e52f9eabcba45e/genome.fa: No such file or directory
  [E::refs_load_fai] Failed to open reference file '/data3/wsx/nxf/work/29/2381d613e63ff7f2e52f9eabcba45e/genome.fa'
  [E::cram_get_ref] Failed to populate reference for id 0
  [E::cram_decode_slice] Unable to fetch reference #0:1418434-1418633

  /data3/wsx/nxf/work/29/2381d613e63ff7f2e52f9eabcba45e/genome.fa: No such file or directory
  [E::refs_load_fai] Failed to open reference file '/data3/wsx/nxf/work/29/2381d613e63ff7f2e52f9eabcba45e/genome.fa'
  [E::cram_get_ref] Failed to populate reference for id 0
  [E::cram_decode_slice] Unable to fetch reference #0:1418484-1418709

  /data3/wsx/nxf/work/29/2381d613e63ff7f2e52f9eabcba45e/genome.fa: No such file or directory
  [E::refs_load_fai] Failed to open reference file '/data3/wsx/nxf/work/29/2381d613e63ff7f2e52f9eabcba45e/genome.fa'
  [E::cram_get_ref] Failed to populate reference for id 0
  [E::cram_decode_slice] Unable to fetch reference #0:1418560-1418792

  /data3/wsx/nxf/work/29/2381d613e63ff7f2e52f9eabcba45e/genome.fa: No such file or directory
  [E::refs_load_fai] Failed to open reference file '/data3/wsx/nxf/work/29/2381d613e63ff7f2e52f9eabcba45e/genome.fa'
  [E::cram_get_ref] Failed to populate reference for id 0
  [E::cram_decode_slice] Unable to fetch reference #0:1418643-1418864

  /data3/wsx/nxf/work/29/2381d613e63ff7f2e52f9eabcba45e/genome.fa: No such file or directory
  [E::refs_load_fai] Failed to open reference file '/data3/wsx/nxf/work/29/2381d613e63ff7f2e52f9eabcba45e/genome.fa'
  [E::cram_get_ref] Failed to populate reference for id 0
  [E::cram_decode_slice] Unable to fetch reference #0:1418715-1418957

  /data3/wsx/nxf/work/29/2381d613e63ff7f2e52f9eabcba45e/genome.fa: No such file or directory
  [E::refs_load_fai] Failed to open reference file '/data3/wsx/nxf/work/29/2381d613e63ff7f2e52f9eabcba45e/genome.fa'
  [E::cram_get_ref] Failed to populate reference for id 0
  [E::cram_decode_slice] Unable to fetch reference #0:1418809-1419032

  /data3/wsx/nxf/work/29/2381d613e63ff7f2e52f9eabcba45e/genome.fa: No such file or directory
  [E::refs_load_fai] Failed to open reference file '/data3/wsx/nxf/work/29/2381d613e63ff7f2e52f9eabcba45e/genome.fa'
  [E::cram_get_ref] Failed to populate reference for id 0
  [E::cram_decode_slice] Unable to fetch reference #0:1418886-1425146

  /data3/wsx/nxf/work/29/2381d613e63ff7f2e52f9eabcba45e/genome.fa: No such file or directory
  [E::refs_load_fai] Failed to open reference file '/data3/wsx/nxf/work/29/2381d613e63ff7f2e52f9eabcba45e/genome.fa'
  [E::cram_get_ref] Failed to populate reference for id 0
  [E::cram_decode_slice] Unable to fetch reference #0:1424998-1455718

  /data3/wsx/nxf/work/29/2381d613e63ff7f2e52f9eabcba45e/genome.fa: No such file or directory
  [E::refs_load_fai] Failed to open reference file '/data3/wsx/nxf/work/29/2381d613e63ff7f2e52f9eabcba45e/genome.fa'
  [E::cram_get_ref] Failed to populate reference for id 0
  [E::cram_decode_slice] Unable to fetch reference #0:1455570-1483018

Work dir:
  /data3/wsx/nxf/work/80/d4633c50cd4e803a872c37d84b9696

Tip: you can replicate the issue by changing to the process work dir and entering the command `bash .command.run`



### Relevant files

_No response_

### System information

OS: CentOS
circdna version: dev
Executor: docker
ShixiangWang commented 1 year ago 
$ ls -alh /data3/wsx/nxf/work/29/2381d613e63ff7f2e52f9eabcba45e/genome.fa
lrwxrwxrwx. 1 wsx wsx 36 Feb 28 15:01 /data3/wsx/nxf/work/29/2381d613e63ff7f2e52f9eabcba45e/genome.fa -> /data1/database/human/hg38/genome.fa
ShixiangWang commented 1 year ago

run $ bash .command.sh successfully while run $ bash .command.run not

DSchreyer commented 1 year ago

That is definitely a weird bug. Did you encounter it again or have you figured out where it could come from ? I am currently bringing out v.1.0.2 and I may be able to include the fix by then.

ShixiangWang commented 1 year ago

@DSchreyer This is because the docker mounts $HOME at default, while the reference path is not located in the default mount path.