Closed Starlento closed 3 months ago
To resolve the "503 Service Unavailable" error when calling xinference and Azure OpenAI text embedding models in your self-hosted (Docker) setup of Dify version 0.6.9, follow these steps:
Check Service Dependencies: Ensure all dependent services (like db
, redis
, weaviate
, etc.) are up and running. The depends_on
directive in your docker-compose.yaml
should help manage this.
Network Configuration: Ensure the services are correctly networked. The ssrf_proxy_network
and default
networks should be properly configured.
Service URLs and Ports: Verify that the service URLs and ports are correctly set and accessible. Ensure that the WEAVIATE_ENDPOINT
and other service endpoints are reachable from the api
and worker
services.
Logs and Debugging: Check the logs of the services to identify any specific issues. Enable more detailed logging by setting LOG_LEVEL
to DEBUG
.
Validate Azure OpenAI Credentials: Ensure the credentials for Azure OpenAI are correctly set. The AzureOpenAITextEmbeddingModel
class in Dify's codebase validates credentials and requires openai_api_base
, openai_api_key
, openai_api_version
, and base_model_name
. Make sure these are correctly configured in your environment variables.
Xinference Configuration: Ensure Xinference is correctly deployed and accessible. Follow these steps to deploy and configure Xinference:
$ pip install "xinference[all]"
$ xinference-local
http://127.0.0.1:9997
, select and deploy the model you need.Running Models
page.Settings > Model Providers > Xinference
, enter the model details and save.Here is a snippet of the docker-compose.yaml
that shows the configuration of the services:
version: '3'
services:
api:
image: langgenius/dify-api:0.6.9
restart: always
environment:
MODE: api
LOG_LEVEL: INFO
SECRET_KEY: sk-9f73s3ljTXVcMT3Blb3ljTqtsKiGHXVcMT3BlbkFJLK7U
DB_HOST: db
DB_PORT: 5432
DB_DATABASE: dify
REDIS_HOST: redis
REDIS_PORT: 6379
VECTOR_STORE: weaviate
WEAVIATE_ENDPOINT: http://weaviate:8080
WEAVIATE_API_KEY: WVF5YThaHlkYwhGUSmCRgsX3tD5ngdN8pkih
depends_on:
- db
- redis
networks:
- ssrf_proxy_network
- default
worker:
image: langgenius/dify-api:0.6.9
restart: always
environment:
MODE: worker
LOG_LEVEL: INFO
SECRET_KEY: sk-9f73s3ljTXVcMT3Blb3ljTqtsKiGHXVcMT3BlbkFJLK7U
DB_HOST: db
DB_PORT: 5432
DB_DATABASE: dify
REDIS_HOST: redis
REDIS_PORT: 6379
VECTOR_STORE: weaviate
WEAVIATE_ENDPOINT: http://weaviate:8080
WEAVIATE_API_KEY: WVF5YThaHlkYwhGUSmCRgsX3tD5ngdN8pkih
depends_on:
- db
- redis
networks:
- ssrf_proxy_network
- default
db:
image: postgres:15-alpine
restart: always
environment:
POSTGRES_PASSWORD: difyai123456
POSTGRES_DB: dify
volumes:
- ./volumes/db/data:/var/lib/postgresql/data
healthcheck:
test: [ "CMD", "pg_isready" ]
interval: 1s
timeout: 3s
retries: 30
redis:
image: redis:6-alpine
restart: always
volumes:
- ./volumes/redis/data:/data
command: redis-server --requirepass difyai123456
healthcheck:
test: [ "CMD", "redis-cli", "ping" ]
weaviate:
image: semitechnologies/weaviate:1.19.0
restart: always
volumes:
- ./volumes/weaviate:/var/lib/weaviate
environment:
QUERY_DEFAULTS_LIMIT: 25
AUTHENTICATION_APIKEY_ENABLED: 'true'
AUTHENTICATION_APIKEY_ALLOWED_KEYS: 'WVF5YThaHlkYwhGUSmCRgsX3tD5ngdN8pkih'
networks:
- ssrf_proxy_network
- default
networks:
ssrf_proxy_network:
driver: bridge
internal: true
default:
driver: bridge
Ensure that the XinferenceTextEmbeddingModel
class is correctly implemented and that the credentials are validated properly. If the issue persists, check the logs for any specific errors and adjust the configurations accordingly [1][2][3][4][5].
To continue the conversation, mention @dosu.
Self Checks
Dify version
0.6.9
Cloud or Self Hosted
Self Hosted (Docker)
Steps to reproduce
Add xinference text embedding model. Drag a pdf to knowledge base.
I also tried azure openai service text-embedding. Failed as well, but the gpt-4o API works. Logically since I add the url successfully, the service should be valid.
✔️ Expected Behavior
successfully call the api and create vector db.
❌ Actual Behavior