search

BerriAI / litellm

Call all LLM APIs using the OpenAI format. Use Bedrock, Azure, OpenAI, Cohere, Anthropic, Ollama, Sagemaker, HuggingFace, Replicate (100+ LLMs)
https://docs.litellm.ai/docs/
Other
10.05k stars 1.12k forks source link

[Feature]: Support wildcard name for model_name in model_list #4238

Open krrishdholakia opened 1 week ago

krrishdholakia commented 1 week ago

The Feature

model_list:
  - model_name: "vertex_ai/*"
    litellm_params:
      model: "vertex_ai/*"
      vertex_project: os.environ/VERTEXAI_PROJECT
  - model_name: "anthropic/*"
    litellm_params:
      model: "anthropic/*"
      api_key: os.environ/ANTHROPIC_API_KEY      
  - model_name: "gemini/*"
    litellm_params:
      model: "gemini/*"
      api_key: os.environ/GEMINI_API_KEY

Motivation, pitch

user request

Twitter / LinkedIn details

cc: @ggallotti

jbohnslav commented 1 day ago

This seems like it should already be possible, e.g. the example config: . However, using the latest docker image, the model shows up as "*", and obviously doesn't work.

krrishdholakia commented 1 day ago

hey @jbohnslav the * implies any openai model should just work - can you try that and let mek now if you see it?

The ask on this ticket is having it work for other providers too