Closed taralika closed 3 weeks ago
Hey @taralika curious - why do you use model_group_alias?
Context - trying to understand our users better
why do you use model_group_alias?
for various reasons:
This is now live on v1.44.21
@krrishdholakia after upgrading to v1.44.21
our health checks have started failing. The issue I think is you're treating "model-group-1"
itself as a model and including "model-group-1"
in list of models with empty model_info and also performing health check on "model-group-1"
. Instead, you should look up all models underneath "model-group-1"
and add them as models under model group "alias-1"
and just perform health check once on these models.
You may want to revert this first and hotfix 1.44.21 as others who have aliases will also start seeing health checks fail.
@taralika when you say health checks do you mean calling /health
?
@taralika when you say health checks do you mean calling
/health
?
yes
Got it - i'll push a fix + add a test for this to our ci/cd flow @taralika
hi @krrishdholakia is this fixed now? I tried v1.44.24 and I can see the "alias-1"
in GET /models
call, and I also see this alias doesn't show up in GET /health
call - so that's great. But what's still missing is that GET /model/info
has no mention of "alias-1"
- as mentioned in description of this ticket, we'll need this so clients know what models are under this alias.
The Feature
When the
proxy-config.yaml
has:Response for
GET /models
should look like this:Response for
GET /model/info
should look like this:Response for
GET /model_group/info
should look like:Motivation, pitch
Users of litellm proxy have no idea what aliases are available, they can only see the model group names. Please expose the aliases at same level as model group names so users of the litellm proxy can take advantage of the aliases as well.
Twitter / LinkedIn details
No response