search

substratusai / lingo

Lightweight ML model proxy and autoscaler for kubernetes
https://www.substratus.ai
Apache License 2.0
96 stars 6 forks source link

Expose vLLM metrics through lingo #85

Open samos123 opened 3 months ago

samos123 commented 3 months ago

https://docs.vllm.ai/en/latest/serving/metrics.html