search

OAID / Tengine

Tengine is a lite, high performance, modular inference engine for embedded device
Apache License 2.0
4.59k stars 993 forks source link

Tengine and TIM-VX utilitze only NPU? #569

Closed janoslim closed 3 years ago

janoslim commented 3 years ago

Hello, I am using Tengine with TIM-VX on vim3 A311d, and I test vgg19 quantized model and its latency is 36ms which is quite fast. And I wonder Tengine use only NPU running model or utilizing cpu gpu npu or cpu npu both on inference time using scheduling or sth. Thank you!

kalcohol commented 3 years ago

Tengine has finished some additional work, so a model could use npu and cpu both. if model not fully supported by npu, Tengine will split it to several parts, some parts will be running on npu, the other will be running on cpu. Tengine also support gpu, see ACL backend for more detials.