Dorado duplex basecalling with modified bases doesn't work/respond

[desmodus1984]

Issue Report

Please describe the issue:

Hi, I was finally able to do sequence my samples and I want to basecalle my data and get, if possible, modified bases. I read that duplex basecalling can get modified bases and I used the following code with no log at all:

nohup /home/juaguila/appz/dorado-0.6.2-linux-x64/bin/dorado duplex --min-qscore 7 -c 50000 --emit-fastq sup,5mC_5hmC Ju760_1-split_by_channel/ > duplex.test.log

I am running on a server with 48 cores, I thought that dorado can automatically detect cores, and the log file was empty.

Steps to reproduce the issue:

I "installed" based on Please list any steps to reproduce the issue.

Then, I did the duplex data preparation with pod5 based on "Improving the Speed of Duplex Basecalling". Then I tried running duplex on the new duplex-pod5 directory, and got no response from Dorado.

Run environment:

• Dorado version: 0.6.2+14a7067
• Dorado command:
• Operating system:
• 3.10.0-1160.105.1.el7.x86_64
• NAME="CentOS Linux" VERSION="7 (Core)" ID="centos" ID_LIKE="rhel fedora" VERSION_ID="7" PRETTY_NAME="CentOS Linux 7 (Core)" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:centos:centos:7" HOME_URL="https://www.centos.org/" BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7" CENTOS_MANTISBT_PROJECT_VERSION="7" REDHAT_SUPPORT_PRODUCT="centos" REDHAT_SUPPORT_PRODUCT_VERSION="7" Icon name: computer-server Chassis: server Machine ID: e86d5e6c9e304b10bfe9eb2c698cee62 Boot ID: d418135d14f44223b92be19e8f50a591 Operating System: CentOS Linux 7 (Core) CPE OS Name: cpe:/o:centos:centos:7 Kernel: Linux 3.10.0-1160.105.1.el7.x86_64 Architecture: x86-64

• Hardware (CPUs, Memory, GPUs):
• CPU 48,
• Memory 400 GB available
• Source data type (e.g., pod5 or fast5 - please note we always recommend converting to pod5 for optimal basecalling performance):
• pod5
• Source data location (on device or networked drive - NFS, etc.): server, assume it is -NFS.
• Details about data (flow cell, kit, read lengths, number of reads, total dataset size in MB/GB/TB): latest flowcell. variable read length, N reads ~ 500k, Dataset -raw 30 GB
• Dataset to reproduce, if applicable (small subset of data to share as a pod5 to reproduce the issue):

Logs

empty

• Please provide output trace of dorado (run dorado with -v, or -vv on a small subset)

[HalfPhoton]

Hi @desmodus1984,

From your command:

nohup dorado duplex --min-qscore 7 --chunksize 50000 --emit-fastq sup,5mC_5hmC Ju760_1-split_by_channel/ > duplex.test.log

There could be a couple of issues:

1. Dorado writes htslib (fastq/bam) outputs to stdout and writes logging output to stderr. To capture stderr logs to a file you must redirect the stderr output using 2> as follows:
   • dorado sup,5mC_5hmC data/ ... > output.fastq 2> duplex.test.log
2. The --chunksize 50_000 is unusually large - and could be causing instability - I would recommend not changing this value and leaving as the default 10_000. This could be why you're getting no fastq data written to duplex.test.log. Do you have a specific reason for setting chunsize to 50k?

Do you have a GPU in this system? Duplex basecalling, on a sup model with mods without a GPU is going to be very slow.

Kind regards, Rich