search

triton-inference-server / server

The Triton Inference Server provides an optimized cloud and edge inferencing solution.
https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/index.html
BSD 3-Clause "New" or "Revised" License
8k stars 1.44k forks source link

Does TIS supports other devices such as google's TPU? #3271

Closed dulvqingyunLT closed 3 years ago

dulvqingyunLT commented 3 years ago

Is your feature request related to a problem? Please describe. if we use some other AI accelerators, but not GPUs, is it possible that we use TIS to do inference serving?

Tabrizian commented 3 years ago

It is possible to use Python backend to run on TPUs. https://github.com/triton-inference-server/python_backend

dyastremsky commented 3 years ago

Closing due to inactivity.