nanoporetech / dorado

Oxford Nanopore's Basecaller
https://nanoporetech.com/
Other
495 stars 59 forks source link

Resume dorado running out of HPC walltime out on nextflow script #734

Closed setshabaTaukobong closed 5 months ago

setshabaTaukobong commented 5 months ago

Good day, I am basecalling using a nextflow script however I keep running out of walltime on our HPC. Ive tried including --resume once I get resources again like so:

/*
 * Basecalling PORE5 files using Dorado
 */
process BASECALL {
    module 'dorado'
    debug true

    publishDir("${params.outdir}/bam_bascall", mode: 'copy')

    input:
    path sample_id

    output:
    path 'sample_id.bam' , emit: bamfiles_complete

    script:
    """
    dorado basecaller /home/staukobong/dna_r10.4.1_e8.2_400bps_hac@v4.2.0 $sample_id --emit-moves --resume > sample_id.bam
    """
}

I however I get an error showing the usage/help message. Could I please get some assistance on how I can resume basecalling from where is it stopped once I have run out of walltime. Thank you

HalfPhoton commented 5 months ago

@setshabaTaukobong You're not correctly setting the <hts_file> to the --resume-from <hts_file> argument which is why you're getting an error from dorado indicating there's an issue with your inputs.

dorado basecaller --help
    --resume-from                 
       Resume basecalling from the given HTS file. 
       Fully written read records are not processed again. [default: ""]

You'll also need to add some logic to determine if there is an existing output sample_id.bam in the working directory and to ensure that it's not overwritten by the next call to Dorado. Something like this (untested):

# Add --resume-from argument if there is an existing output 
RESUME=""
if [ -f "sample_id.bam" ]; then
  # Rename old file so that it's not overwritten in the subsequent call to dorado
  mv sample_id.bam sample_id.previous.bam
  RESUME=" --resume-from sample_id.previous.bam"
fi
dorado basecaller model $sample_id --emit-moves \$RESUME > sample_id.bam

Kind regards, Rich

HalfPhoton commented 5 months ago

@setshabaTaukobong, has this resolved your issue?

setshabaTaukobong commented 5 months ago

Ah makes sense. Thank you for your help!