Closed majestichou closed 2 weeks ago
To increase the concurrency of Dify for Q&A usage, you can adjust several environment variables related to rate limits and resource allocation. Here are the key variables to modify:
SERVER_WORKER_AMOUNT: This sets the number of server workers. The recommended formula is CPU core count x 2 + 1
. Increasing the number of workers can help handle more concurrent requests.
SERVER_WORKER_CLASS: The default is gevent
. For Windows, you can switch to sync
or solo
.
GUNICORN_TIMEOUT: This is the request processing timeout. The default is 200 seconds, but setting it to 360 seconds is recommended to support long SSE connections.
CELERY_WORKER_AMOUNT: This sets the number of Celery workers. The default is 1, but you can increase it as needed.
SQLALCHEMY_POOL_SIZE: This sets the size of the database connection pool. The default is 30 connections, but you can increase it to handle more concurrent database connections.
Additionally, ensure that your API keys (e.g., OpenAI API key, embedding model API key) have higher rate limits by upgrading your plans or contacting the providers.
By adjusting these settings, you can increase the concurrency of Dify for Q&A usage [1][2][3].
To continue talking to Dosu, mention @dosu.
Self Checks
Provide a description of requested docs changes
I hope an answer to this question will be added to the official documentation: How to increase the concurrency of dify so that more people can use dify for Q&A at the same time? What environment variables can I change to achieve this? maybe increase "SERVER_WORKER_AMOUNT", increase "CELERY_WORKER_AMOUNT"? I am not sure how to achieve my goal.