[Feature Request]: aichat --update-models based on openai-compatible /v1/models

[barakplasma]

Is your feature request related to a problem? Please describe.

I'm frustrated by needing to add new models by hand in the hours after new models are released.

Describe the solution you'd like

I'd like a new command for adding all the models supported by my API key for a given openai-compatible service. For example: aichat --update-models which would GET the openai-compatible API /v1/models available to a given API KEY and transform them to add more models to the models.yml or config.yml .

The following request works on the API_BASE for openai, Mistral, Nvidia nim, groq, etc.

curl https://api.openai.com/v1/models \                                       
  -H "Authorization: Bearer $OPENAI_API_KEY"

returns:

{
  "object": "list",
  "data": [
    {
      "id": "gpt-4-turbo",
      "object": "model",
      "created": 1712361441,
      "owned_by": "system"
    },
    {
      "id": "gpt-4o-mini-2024-07-18",
      "object": "model",
      "created": 1721172717,
      "owned_by": "system"
    }
]
}

for Mistral it even includes a bit more detail:

[{
      "id": "ministral-3b-latest",
      "object": "model",
      "created": 1729110798,
      "owned_by": "mistralai",
      "name": "ministral-3b-2410",
      "description": "Official ministral-3b-2410 Mistral AI model",
      "max_context_length": 32768,
      "aliases": [
        "ministral-3b-2410"
      ],
      "deprecation": null,
      "capabilities": {
        "completion_chat": true,
        "completion_fim": false,
        "function_calling": true,
        "fine_tuning": false,
        "vision": false
      },
      "type": "base"
    }]

for groq:

[{
      "id": "llama-3.2-90b-text-preview",
      "object": "model",
      "created": 1727285716,
      "owned_by": "Meta",
      "active": true,
      "context_window": 8192,
      "public_apps": null
    }]

for Nvidia, doesn't even require an API KEY

$ curl https://integrate.api.nvidia.com/v1/models
{"object":"list","data":[{"id":"01-ai/yi-large","object":"model","created":735790403,"owned_by":"01-ai"},{"id":"abacusai/dracarys-llama-3.1-70b-instruct","object":"model","created":735790403,"owned_by":"abacusai"}...

It would be great for aichat to take the results and update it's own models.yml or config.yaml with the given id's.

Describe alternatives you've considered

I can always patch / add model names by hand, but it seems like a waste of time. Sometimes I can update to the latest version of aichat to get the new models.yml which also works well.

Additional context

I could write a PR for this if interested. Or provide a script outside of aichat to update config.yml . Also happy for someone else to implement it.

[barakplasma]

This one liner generates the format needed to copy/paste into the config.yml in the models section

curl https://integrate.api.nvidia.com/v1/models | jq '[{name: .data[].id}]'

the results can be pasted into config.yml like so

- type: openai-compatible
  name: nim
  models: [
    {
      "name": "01-ai/yi-large"
    },
    {
      "name": "abacusai/dracarys-llama-3.1-70b-instruct"
    }
  ]
  api_base: https://integrate.api.nvidia.com/v1
  api_key: nvapi-...

[sigoden]

We will not support this feature.

The model list from /v1/models only contains the name field. The model list includes a mix of chat, embedding, and reranker models that need to be distinct. AIChat cannot use them directly.

[khalido]

Thanks for the explanation, it makes sense why this issue was closed, I have a related request which delivers part of the original request:

It seems providers are slowly adopting a -latest model name convention. Two examples:

• Anthropic: claude-3-5-sonnet-latest (aichat is on June 24 release, -latest uses Oct release)
• OpenAI: chatgpt-4o-latest (already available in aichat)

Add the -latest model where possible, this makes it easier to use/try out updated models without requiring a aichat update.