iridium-soda
commented
1 month ago Also, here is my custom models_config.yaml:
model_providers:
############################ Local ############################################
# Uncomment this provider if you want to use local models providers #
# using ollama and infinity model server #
###############################################################################
# - provider_name: local-ollama
# api_format: openai
# base_url: http://ollama-server:11434/v1/
# api_key_env_var: ""
# llm_model_ids:
# - "qwen2:1.5b"
# embedding_model_ids: []
# reranking_model_ids: []
# default_headers: {}
# - provider_name: local-infinity
# api_format: openai
# base_url: http://infinity-server:7997/
# api_key_env_var: INFINITY_API_KEY
# llm_model_ids: []
# embedding_model_ids:
# - "mixedbread-ai/mxbai-embed-large-v1"
# reranking_model_ids:
# - "mixedbread-ai/mxbai-rerank-xsmall-v1"
# default_headers: {}
# - provider_name: faster-whisper
# api_format: openai
# base_url: http://faster-whisper:8000
# api_key_env_var: ""
# llm_model_ids: []
# embedding_model_ids: []
# reranking_model_ids: []
# audio_model_ids:
# - "Systran/faster-distil-whisper-large-v3"
# default_headers: {}
############################ OpenAI ###########################################
# Uncomment this provider if you want to use OpenAI as a models provider #
# Remember to set `OPENAI_API_KEY` in container environment #
###############################################################################
# - provider_name: openai
# api_format: openai
# api_key_env_var: OPENAI_API_KEY
# llm_model_ids:
# - "gpt-3.5-turbo"
# - "gpt-4o"
# embedding_model_ids:
# - "text-embedding-3-small"
# - "text-embedding-ada-002"
# reranking_model_ids: []
# default_headers: {}
############################ TrueFoundry ###########################################
# Uncomment this provider if you want to use TrueFoundry as a models provider #
# Remember to set `TFY_API_KEY` in container environment #
####################################################################################
# - provider_name: truefoundry
# api_format: openai
# base_url: https://llm-gateway.truefoundry.com/api/inference/openai
# api_key_env_var: TFY_API_KEY
# llm_model_ids:
# - "openai-main/gpt-4o-mini"
# - "openai-main/gpt-4-turbo"
# - "openai-main/gpt-3-5-turbo"
# embedding_model_ids:
# - "openai-main/text-embedding-3-small"
# - "openai-main/text-embedding-ada-002"
# reranking_model_ids: []
# default_headers: {}
############################ Mixedbread ###########################################
# Uncomment this provider if you want to use Mixedbread as a models provider #
# Remember to set `MIX_API_KEY` in container environment (./compose.env) #
#####################################################################################
- provider_name: Mixedbread
api_format: openai
base_url: https://api.mixedbread.ai
api_key_env_var: MIX_KEY
llm_model_ids: []
embedding_model_ids:
- 'mixedbread-ai/mxbai-embed-large-v1'
reranking_model_ids:
- "mixedbread-ai/mxbai-rerank-large-v1"
default_headers: {}
############################ Azure ###########################################
# Uncomment this provider if you want to use Azure as a models provider #
# Remember to set `AZURE_API_KEY` in container environment #
###############################################################################
- provider_name: Azure
api_format: openai
base_url: AZURE_ENDPOINT
api_key_env_var: AZURE_KEY
llm_model_ids:
- 'gpt35-16k'
embedding_model_ids: []
reranking_model_ids: []
default_headers: {}
FabrizioDG
chiragjn
mnvsk97
I’m trying to use Azure OpenAI as my LLM provider and the Mixedbread Remote API for reranking and embedding. I’m attempting to run the service using Docker Compose, but I’m encountering an issue preventing the service from starting.
I’m unsure whether there’s a configuration mistake on my end, or if the issue is related to environment variables. Specifically, I’m wondering if the TFY_API_KEY must be explicitly assigned for the service to run correctly.
Questions:
Are there any specific configurations or common pitfalls when integrating Azure OpenAI with Mixedbread API through Docker Compose? Is the
TFY_API_KEYrequired for the deployment of the service? Could there be any other setup steps that I might have overlooked? Thank you for your assistance!Here is my error message when trying to deploy: