search

nanoporetech / dorado

Oxford Nanopore's Basecaller
https://nanoporetech.com/
Other
477 stars 59 forks source link

Can TensorRT bring greater performance improvements? #680

Closed keithyin closed 5 months ago

keithyin commented 5 months ago

Hi everyone, I'm currently learning about Dorado and I see that it uses libtorch for inference. Have you tried using TensorRT? From the documentation, it seems that TensorRT can bring better speed improvements.

iiSeymour commented 5 months ago

Hi @keithyin in Dorado we have custom CUDA layers for our specific needs which are more heavily optimised than can be achieved with frameworks such as TensorRT.