search

triton-inference-server / server

The Triton Inference Server provides an optimized cloud and edge inferencing solution.
https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/index.html
BSD 3-Clause "New" or "Revised" License
8.18k stars 1.46k forks source link

Conda Package for Inference Server #6940

Open iamthebot opened 7 months ago

iamthebot commented 7 months ago

Is your feature request related to a problem? Please describe. It would be handy to be able to install optimize builds of the inference server via conda (via either the nvidia or conda-forge channels).

Currently, only the python client is published.

Happy to take a stab at this unless NVIDIA has concrete plans to work on this already?

Describe the solution you'd like A clear and concise description of what you want to happen.

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

Additional context Add any other context or screenshots about the feature request here.

yinggeh commented 7 months ago

Have you taken a look at pytriton? It might be helpful in your case. https://github.com/triton-inference-server/pytriton

iamthebot commented 7 months ago

@yinggeh I have. But this issue is for the server binary itself. Eg; if one wants to deploy triton inference server without using the NVIDIA docker images. Installation via conda makes that very easy.

yinggeh commented 7 months ago

@iamthebot Thanks. I will pass your feedback to the team.

yinggeh commented 7 months ago

DLIS-6303