Error: "An exception occurred in stage 3: KeyError - 'barcode_full_arrangement'"

[Timothy-Amos]

Hey James,

Here's the details of the errors I got with a barcoded sample.

The (nextflow log of the) command was:

tga561@gadi-login-02:3f4c0f6b99cebe80b496a687b7dbff:$ cat .command.sh
#!/bin/bash -ue
if [[ dna_r10.4.1_e8.2_400bps_5khz_hac.cfg == *modbases* ]]; then
    CALLMODS="--call_mods"
    OUTPUT="reads.sam"
else
    CALLMODS=""
    OUTPUT="reads.fastq"
fi
if [[ SQK-NBD114-24 != NONE ]]; then
    BARCODE_KITS="--barcode_kits SQK-NBD114-24"
else
    BARCODE_KITS=""
fi
/usr/bin/time -v buttery-eel -i PGAXHN000007_reads.blow5 -o $OUTPUT -g /g/data/if89/apps/buttery-eel/0.5.0+dorado7.4.12/ont-dorado-server/bin/ --port auto --use_tcp --config dna_r10.4.1_e8.2_400bps_5khz_hac.cfg -x cuda:all -q 9 --seq_sum $CALLMODS $BARCODE_KITS --slow5_threads 10 --slow5_batchsize 100 --procs 20
echo "basecalling success"

And the errors are:

tga561@gadi-login-02:3f4c0f6b99cebe80b496a687b7dbff:$ grep Error .command.log | sort | uniq -c
     57 An exception occurred in stage 3: KeyError - An exception occurred in stage 3: KeyError - 'barcode_full_arrangement'
  13958 An exception occurred in stage 3: KeyError - 'barcode_full_arrangement'

Full log is:

/scratch/ox63/tga561/tmp/PGAXHN000007/work/33/3f4c0f6b99cebe80b496a687b7dbff/.command.log

Some settings details:

tga561@gadi-login-02:3f4c0f6b99cebe80b496a687b7dbff:$ head -11 .command.log
Loading /g/data/if89/apps/modulefiles/buttery-eel/0.5.0+dorado7.4.12
  Loading requirement: intel-mkl/2021.4.0 python3/3.10.4

               ~  buttery-eel - SLOW5 Guppy/Dorado Server Basecalling  ~
                            version: 0.5.0
==========================================================================
  ARGS
==========================================================================
args:
 Namespace(input='PGAXHN000007_reads.blow5', output='reads.fastq', basecaller_bin=PosixPath('/g/data/if89/apps/buttery-eel/0.5.0+dorado7.4.12/ont-dorado-server/bin'), config='dna_r10.4.1_e8.2_400bps_5khz_hac.cfg', call_mods=False, qscore=9, slow5_threads=10, procs=20, slow5_batchsize=100, quiet=False, max_read_queue_size=20000, log='buttery_basecaller_logs', moves_out=False, trim_adapters=False, seq_sum=True, barcode_kits=['SQK-NBD114-24'], enable_trim_barcodes=False, require_barcodes_both_ends=False, duplex=False, single=False, profile=False, do_read_splitting=True, detect_adapter=False, detect_mid_strand_adapter=False, detect_mid_strand_barcodes=False, above_7310=True, above_7412=True)
['--port', 'auto', '--use_tcp', '-x', 'cuda:all']

Thanks.

James' immediate response:

"the only reason this would fail, is because ONT modified the fields and I didn't see it I'll have to modify that list comprehension to

bc_sum_out = "\t".join([bcalled_read["parent_read_id"]]+[bcalled_read["read_id"]]+[str(call['metadata'].get(i, ".") for i in bc_keys])

so it defaults to . if it can't find the field Not really a solution, but it stops the failure"

[Psy-Fer]

Excellent. Thanks for posting this Tim.

I'll work on a fix this arvo.

James

[Psy-Fer]

This should now be fixed in v0.5.1

https://github.com/Psy-Fer/buttery-eel/releases/tag/v0.5.1

Cheers, James