deepjavalibrary / djl

An Engine-Agnostic Deep Learning Framework in Java
https://djl.ai
Apache License 2.0
4.16k stars 661 forks source link

I have a pytorch model exported as an onnx format, Which engine should I to set when I use Criteria? During prediction, I use NDManager.newBaseManager to get a NDManager, which engine should NDManager.newBaseManager use? #3508

Open BlackRen09 opened 3 weeks ago

BlackRen09 commented 3 weeks ago

I have a pytorch model used for Text-to-Speech, the encoder and decoder are exported as onnx format. When Loading these model file, Criteria has a param named engine, I wonder which engine (OnnxRuntime or pytorch) should be used? After that, outputs from the encoder will be as inputs to the decoder, For managing NDResource, I have to get a NDManager to manage temporal resources, so I use NDManager.newBaseManager, when I call this function, which engine should be used? (I have read the source of NDManager, different engine will get different implement of manager).

BlackRen09 commented 3 weeks ago

If I want to use engine profiler support, OnnxRuntime way or choose pytorch way? Thanks for your answering!