search

triton-inference-server / server

The Triton Inference Server provides an optimized cloud and edge inferencing solution.
https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/index.html
BSD 3-Clause "New" or "Revised" License
7.82k stars 1.42k forks source link

How to access the triton inference server from outside of the k8s cluster through ingressroutes? #7079

Open zengqingfu1442 opened 3 months ago

zengqingfu1442 commented 3 months ago

Since the ingressroutes(https://github.com/triton-inference-server/server/blob/main/deploy/k8s-onprem/templates/ingressroute.yaml) has been deployed as LB to balance requests across all triton pods. How can i access the triton inference server api)http or grpc or in-process-all) through ingressroutes? Should i install traefik ingress controller and then install an ingress?

zengqingfu1442 commented 3 months ago

I can use ingress-nginx as ingress controller and then create an ingress to access triton http/grpc api. This needs to add a dns cname map.