simonw / llm

Access large language models from the command-line
https://llm.datasette.io
Apache License 2.0
4.75k stars 263 forks source link

Ability to configure attachment support for models in extra-openai-models.yaml #602

Open NightMachinery opened 1 week ago

NightMachinery commented 1 week ago

In ~/Library/Application Support/io.datasette.llm/extra-openai-models.yaml, I have this:

- model_id: or:c35s
  model_name: anthropic/claude-3.5-sonnet:beta
  api_base: "https://openrouter.ai/api/v1"
  api_key_name: openrouter

When I try to use -a some_image.png, I get:

Error: This model does not support attachments, but some were provided
❯ llm --version
llm, version 0.17.1

PS: It works with gemini-1.5-pro-latest accessed directly from Google. The problem is probably with the OpenRouter indirection.

simonw commented 1 week ago

From #593:

I added custom gpt-4o azure model through extra-openai-models.yaml and Api proxy config is

- model_id: 4o-azure
  model_name: azure-gpt4o
  api_base: "http://10.41.9.156:4000/"
  api_key_name: azure-gpt4o

then command returns

"describe this image" \
      -a IMG_20240908_161441.jpg
Error: This model does not support attachments, but some were provided

Is there option to extra models that indicate model supports attachments or any other function?