sigoden / aichat

All-in-one AI CLI tool that integrates 20+ AI platforms, including OpenAI, Azure-OpenAI, Gemini, Claude, Mistral, Cohere, VertexAI, Bedrock, Ollama, Ernie, Qianwen, Deepseek...
Apache License 2.0
3.27k stars 217 forks source link

Handle embeddings from Azure OpenAI #668

Closed tomsib2001 closed 3 days ago

tomsib2001 commented 3 days ago

There seems to be an issue using embeddings provided as part of the Azure OpenAi platform I can access them using python

client = AzureOpenAI(
    api_key="[API_KEY]",
    azure_endpoint=my_endpoint,
    api_version="2023-12-01-preview"
)

response = client.embeddings.create(input=[text], model="text-embedding-ada-002-2")

But when I try to add it to my config.yaml file:

  - type: azure-openai
    name: azure_ai
    api_base: [URL]
    api_key: [API_KEY]
    models:
      - name: gpt-4o-e # 128k
        max_input_tokens: 128000
        max_output_tokens: 4096
        supports_function_calling: true
        supports_vision: true
[...]
      - name: text-embedding-3-large
        mode: embedding
      - name: text-embedding-3-small
        mode: embedding
      - name: text-embedding-ada-002-2
        mode: embedding

and then

thomas@L15-77:$ aichat --rag myrag
? Select embedding model:
> openai:text-embedding-3-large (dimension:3072; max-tokens:8191; price:0.13; batch:100)
  openai:text-embedding-3-small (dimension:1536; max-tokens:8191; price:0.02; batch:100)
[↑↓ to move, enter to select, type to filter]

(I only see openai embeddings, not azure ones)

Expected behavior The azure_openai embedding models should appear.

Configuration

model                   azure_ai:gpt-4o-e
max_output_tokens       -
temperature             -
top_p                   -
dry_run                 false
save                    true
keybindings             emacs
wrap                    no
wrap_code               false
save_session            -
compress_threshold      50000
function_calling        true
rag_reranker_model      -
rag_top_k               4
highlight               true
light_theme             false
config_file             /home/thomas/.config/aichat/config.yaml
roles_file              /home/thomas/.config/aichat/roles.yaml
functions_dir           /home/thomas/.config/aichat/functions
agents_functions_dir    /home/thomas/.config/aichat/functions/agents
agents_config_dir       /home/thomas/.config/aichat/agents
rags_dir                /home/thomas/.config/aichat/rags
sessions_dir            /home/thomas/.config/aichat/sessions
messages_file           /home/thomas/.config/aichat/messages.md

Environment (please complete the following information):

tomsib2001 commented 3 days ago

Note that ollama embeddings from a remote location don't seem to be handled either, although I managed to make them work locally on a different computer.

sigoden commented 3 days ago

miss type: embedding

https://github.com/sigoden/aichat/blob/8b8f8b37c35479f747a70a65f069c0d8d4b01dcf/config.example.yaml#L98-L102

tomsib2001 commented 3 days ago

Thanks a lot for the quick response! I don't know where this "mode" keyword came from, maybe a previous version ? In any case that solved both problems, for Azure and Ollama