search

langgenius / dify

Dify is an open-source LLM app development platform. Dify's intuitive interface combines AI workflow, RAG pipeline, agent capabilities, model management, observability features and more, letting you quickly go from prototype to production.
https://dify.ai
Other
43.12k stars 5.98k forks source link

How to increase the concurrency of dify so that more people can use dify for Q&A at the same time？ #5722

Closed majestichou closed 2 weeks ago

majestichou commented 2 months ago

Self Checks

Provide a description of requested docs changes

I hope an answer to this question will be added to the official documentation： How to increase the concurrency of dify so that more people can use dify for Q&A at the same time？ What environment variables can I change to achieve this? maybe increase "SERVER_WORKER_AMOUNT", increase "CELERY_WORKER_AMOUNT"? I am not sure how to achieve my goal.

dosubot[bot] commented 2 months ago

To increase the concurrency of Dify for Q&A usage, you can adjust several environment variables related to rate limits and resource allocation. Here are the key variables to modify:

  1. SERVER_WORKER_AMOUNT: This sets the number of server workers. The recommended formula is CPU core count x 2 + 1. Increasing the number of workers can help handle more concurrent requests.

  2. SERVER_WORKER_CLASS: The default is gevent. For Windows, you can switch to sync or solo.

  3. GUNICORN_TIMEOUT: This is the request processing timeout. The default is 200 seconds, but setting it to 360 seconds is recommended to support long SSE connections.

  4. CELERY_WORKER_AMOUNT: This sets the number of Celery workers. The default is 1, but you can increase it as needed.

  5. SQLALCHEMY_POOL_SIZE: This sets the size of the database connection pool. The default is 30 connections, but you can increase it to handle more concurrent database connections.

Additionally, ensure that your API keys (e.g., OpenAI API key, embedding model API key) have higher rate limits by upgrading your plans or contacting the providers.

By adjusting these settings, you can increase the concurrency of Dify for Q&A usage [1][2][3].

To continue talking to Dosu, mention @dosu.