samtools: files already existing

[smoe]

Hello,

[Fri Dec 10 15:02:37 2021]
[Fri Dec 10 15:02:37 2021]
Error in rule sort_bam:
Error in rule sort_bam:
    jobid: 101
    jobid: 89
    output: .../output/mapped_reads/158366_Aligned.sortedByCoord.out.bam
    output: .../output/mapped_reads/158342_Aligned.sortedByCoord.out.bam
    log: .../output/logs/samtools_sort_158366.log (check log file(s) for error message)
    log: .../output/logs/samtools_sort_158342.log (check log file(s) for error message)
    shell:
        /gnu/store/qla8q0cjjfdy1j6mjpqdcxybm7b3dipv-samtools-1.11/bin/samtools  sort -o .../output/mapped_reads/158366_Aligned.sortedByCoord.out.bam .../output/mapped_reads/158366_Aligned.out.bam >> .../output/logs/samtools_sort_158366.log 2>&1
        (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)
    shell:
        /gnu/store/qla8q0cjjfdy1j6mjpqdcxybm7b3dipv-samtools-1.11/bin/samtools  sort -o .../output/mapped_reads/158342_Aligned.sortedByCoord.out.bam .../SamplesRabbit/output/mapped_reads/158342_Aligned.out.bam >> .../output/logs/samtools_sort_158342.log 2>&1
        (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

The error is that temporary files that should not exist are existing

$ cat /mariner/projects/HumanRabbitTFibrosisTimeSeries/SamplesRabbit/output/logs/samtools_sort_158366.log
[E::hts_open_format] Failed to open file "/mariner/projects/HumanRabbitTFibrosisTimeSeries/SamplesRabbit/output/mapped_reads/158366_Aligned.sortedByCoord.out.bam.tmp.0000.bam" : File exists
samtools sort: failed to create temporary file "/mariner/projects/HumanRabbitTFibrosisTimeSeries/SamplesRabbit/output/mapped_reads/158366_Aligned.sortedByCoord.out.bam.tmp.0000.bam": File exists

My hunch is that I cancelled / ran into a memory issue with my previous run. Maybe there could be some magic with "find .. -delete" introduced for a prior cleanup?

[borauyar]

@smoe maybe this would help? https://stackoverflow.com/questions/47889137/snakemake-go-back-and-clean-up-temp-files see --delete-temp-output option of snakemake >5.0

[smoe]

Looks good. I cannot reproduce this atm.

[borauyar]

Okay, thanks!