[Question] How does this compare to Triton Inference Server ?

[MatthieuToulemont]

Just a random thought, how does this compare to other serving frameworks like Triton Inference Server, Torch Serve ... ?

Thanks for this great project :)

[JoanFM]

Hello @MatthieuTPHR,

This is an interesting question.

Jina has some similarities with these projects as it can also be used to serve any Machine Learning models, but the main difference of Jina is the abstractions and flexibility on which it is built.

For instance:

• DocumentArray lets the user reason about their data in a multimodal way in a very handy API
• Executors allow anyone to turn their own DocArray processing into a microservice working on DocumentArrays
• Flows let you combine these Executors into complex pipelines to power your applications.
• Then Jina also helps you turn this into production ready services to run in Kubernetes

These Executors could be dedicated to serve Machine Learning models but they could serve many different purposes. Other frameworks are simply specialized in ML models and you normally need to adapt to them.

I hope this clarifies a little your question.

Thanks for asking and looking forward for the feedback.