search

nanoporetech / taiyaki

Training models for basecalling Oxford Nanopore reads
https://nanoporetech.com/
Other
115 stars 42 forks source link

Error with guppy using a new model trained with taiyaki #115

Open jlaroche opened 3 years ago

jlaroche commented 3 years ago

I trained a new model with taiyaki and export it with dump_json.py I try to use it with guppy with this cmd:

guppy_basecaller -i subset_validation_single_fast5 -s subset_validation_basecall_new -c /prg/guppy/4.4.1/data/dna_r9.4.1_450bps_hac.cfg -m model_laval1.json --device cuda:0

I got this in the log file:

2021-06-02 16:17:05.873911 [guppy/message] ONT Guppy basecalling software version 4.4.1+1c81d62 config file: /prg/guppy/4.4.1/data/dna_r9.4.1_450bps_hac.cfg model file: /home/jelar5/cam/samples/neg_ctrl/guppy_training/taiyaki_model/model_laval1.json input path: /project/bioinfo/users/jelar5/consultations/camila_lima/samples/neg_ctrl/subset_validation_single_fast5 save path: /project/bioinfo/users/jelar5/consultations/camila_lima/samples/neg_ctrl/subset_validation_basecall_new chunk size: 2000 chunks per runner: 512 records per file: 4000 num basecallers: 4 gpu device: cuda:0 kernel path: runners per device: 4 2021-06-02 16:17:05.874334 [guppy/info] crashpad_handler not supported on this platform. 2021-06-02 16:17:12.314632 [guppy/info] CUDA device 0 (compute 6.0) initialised, memory limit 17071734784B (16804216832B free) 2021-06-02 16:17:12.339841 [guppy/error] Could not open gru_384_384_1_1_1_0.fatbin 2021-06-02 16:17:12.340052 [guppy/warning] Failed to load gru(gru_384_384_1_1_1_0) from fatbin 2021-06-02 16:17:12.342134 [guppy/error] Could not open CUDA kernel file: gru.cu 2021-06-02 16:17:12.342253 [guppy/warning] An error has occurred. Aborting.

Thanks!

Jerome

smaegol commented 3 years ago

I have the same problem. Trained with RNA samples. Is there any solution? Basecalling on CPU is time-consuming...

tmassingham-ont commented 3 years ago

Hello. I'm afraid Guppy only supports a limited number of layer types and sizes and, unfortunately, a size 384 GRU layer is not one of them (the error you are seeing). Size 256 GRU or size size 384 LSTM layers are supported.

Alternatively you could try the basecaller.py script from Taiyaki -- it is generally much slower than Guppy but does support all layers on the GPU. Depending on GPU memory available, increasing the --jobs flag can considerably improve performance.