search

triton-inference-server / server

The Triton Inference Server provides an optimized cloud and edge inferencing solution.
https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/index.html
BSD 3-Clause "New" or "Revised" License
8.07k stars 1.45k forks source link

Add environment variable that allows you to append a prefix to all HTTP requests #7426

Open HeeebsInc opened 2 months ago

HeeebsInc commented 2 months ago

A lot of API tools that are opensource allow you to configure a prefix for the routes. This is needed for more customization in cloud environments. For example, if I had a DNS called myapp.com, and an ingress controller that mapped /triton to the triton pods, not having a prefix parameter will break the deployment.

What im proposing is a parameter that will globally append a prefix to the URLs. For example, the prefix tritonserver --http-url-prefix=triton, will cause the /v2/health/ready to become /triton/v2/health/ready (as well as every other route exposed by the server)

This is similar to the root_path parameter allowed by FastAPI

rmccorm4 commented 1 month ago

This seems like a reasonable and relatively simple request to me. @GuanLuo @nnshah1 what do you think?