Closed fgreinacher closed 1 month ago
BTW: Would be happy to contribute this if the maintainers agree.
hey @fgreinacher we currently provide this info to help debug the error.
Do you want to control this verbosity with a flag? Open to suggestions on this
We usually expose health endpoints
are you trying to expose the health of the proxy or the health of the LLM's
for the health of the proxy, you could use the /health/readiness
or /health/liveliness
endpoint
are you trying to expose the health of the proxy or the health of the LLM's
We want to use this for monitoring the health of the downstream LLMs.
for the health of the proxy, you could use the /health/readiness or /health/liveliness endpoint
Yes, we already use these for Kubernetes probes 👍
Do you want to control this verbosity with a flag? Open to suggestions on this
Yes, that would be the idea. I'll draft a PR and ping you!
great
The Feature
The
/health
endpoint provides a lot of detailed information in thehealthy_endpoints
/unhealthy_endpoints
properties. We usually expose health endpoints without authentication and therefore prefer to keep the response minimal.Would you be open to a new setting like
minimal_health_response
to reduce the amount of data to what people can observe from the outside?Motivation, pitch
This is too much for our taste (in this case
api_base
anderror
, but also other potentially exposed LiteLLM params):Twitter / LinkedIn details
No response