Lightning-AI / LitServe

Lightning-fast serving engine for any AI model of any size. Flexible. Easy. Enterprise-scale.
https://lightning.ai/docs/litserve
Apache License 2.0
2.51k stars 160 forks source link

Add custom path option for healthcheck #341

Closed akuma12 closed 1 week ago

akuma12 commented 1 month ago

🚀 Feature

I'd like to be able to customize the path for the health check. Currently it defaults to /health, but I'd like to be able to add my own prefix to that, such as /my_service/health or any arbitrary value.

Motivation

We use a shared load balancer with path-based routing, so we have to use app-specific prefixes for our paths, such as /my_service_1/foo and /my_service_2/bar. Because of this, the health check path also needs to start with that prefix.

Pitch

Ideally I could add a healthcheck_path to the LitServer class initialization that would override the default path of /health.

Alternatives

Currently I'm getting around this by using server.app.add_api_route, and adding a custom healthcheck method to my LitAPI class. In this method, I've tried using Requests to make a GET request to the local /health endpoint, but that actually causes LitServe to exit, so at the moment it just returns an HTTP 200 response, and doesn't actually check the readiness of the server.

ktrapeznikov commented 1 month ago

Good feature to add

bhimrazy commented 2 weeks ago

Seems like a good feature to add. Even incase of sagemaker, it required a different endpoint for the health check i.e '/ping'.