Open bobanj opened 2 months ago
Are the different models served by one or multiple BentoML servers? If multiple servers, shouldn't they be running on different ports? If the same server, the models should be sharing the same health check path, too. Is my understanding correct?
We are using a diff infra setup (no EKS), hence the reason for asking for this.
@frostming Do you think the health check and metrics prefix is best exposed as a configuration field?
Sorry I didn't get the point. The OP seems to ask for multiple metrics endpoints instead of custom metrics endpoint, but the former doesn't make sense to me.
Not to have multiple metrics endpoints but for the current /metrics path configurable, basically to have something like https://github.com/bentoml/BentoML/blob/0666d62dcaabc9a9a84121c0429e94ae41940733/src/bentoml/_internal/server/http/instruments.py#L87
if scope["path"] == metrics_path_from_config || "/metrics"
Feature request
Would like to be able to provide something like a
prefix_path
in the configuration so themetrics
,healthz
endpoints are available withmy/prefix/path
Motivation
Scenario where multiple models are available under one hostname, ex:
my_host/model_1/endpoint
my_host/model_2/endpoint
Would like to be able to access metrics and healthz endpoints without the need to define custom middleware or define additional healthz action (currently in code as far as can be seen, the comparisons are hardcoded ex:== "/metrics". Would like to be able to setup the accessability for these endpoints by defining somehting like
prefix_path`:my_host/model_1/metrics
my_host/model_1/healthz
my_host/model_2/metrics
my_host/model_2/healthz
Other
No response