Dify - Knowledge Base - Status Queuing

[PubliBrasil]

Self Checks

• [X] This is only for bug report, if you would like to ask a question, please head to Discussions.
• [X] I have searched for existing issues search for existing issues, including closed ones.
• [X] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [X] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
• [X] Please do not modify this template :) and fill in all the required fields.

Dify version

0.7.1

Cloud or Self Hosted

Cloud

Steps to reproduce

I started to encounter problems when uploading any file to the knowledge base to Cloud Dify, manually or via API. I checked and found no limitations on the account.

✔️ Expected Behavior

No response

❌ Actual Behavior

No response

[dosubot[bot]]

Hello, @PubliBrasil! I'm here to assist you with any bugs, questions, or contributions regarding Dify's Knowledge Base. Let's take a look at the issue you've encountered.

I found similar issues and discussions related to files being stuck in the "Queuing" status when uploading to the knowledge base in Dify:

1. Issue #7263 (closed): The suggested solution was to check if the rate limit has been reached for the Embedding model API key used.
2. Issue #5184 (closed): The solution involved checking document limits, subscription constraints, and ensuring the worker service is running correctly.
3. Issue #696 (closed): The problem was related to insufficient OpenAI API key quotas, causing constant retries and blocking queue consumption.

For your specific case in Cloud Dify, the most likely cause is that the rate limit has been reached for the Embedding model API key used ^[1].

To resolve this, you should:

1. Verify the API key's rate limits and ensure they are not exceeded.
2. Check if there are any other constraints or limits being hit, such as document limits or subscription constraints.
3. Ensure that all necessary services, including the Celery worker service, are running correctly.

If the issue persists, you may need to review the logs for any errors or warnings that could provide more insight into why the files are stuck in the "Queuing" status.

To continue talking to Dosu, mention @dosu.

[PubliBrasil]

@dosu I have already checked Dify's limits, as shown in the image I sent previously, and they were not reached. I analyzed the Embedding Model API - Cohere - which is paid - I also did not reach the limit.

[dosubot[bot]]

If you've confirmed that neither Dify's limits nor the Embedding Model API limits have been reached, there are a few other potential causes to investigate:

1. Check for Specific Error Messages or Logs: Review the logs for any specific error messages that might indicate the cause of the issue. Common error messages include:

   • provider_not_initialize: Embedding model not configured
   • provider_quota_exceeded: Model quota exceeded
   • unsupported_file_type: Unsupported file type
   • too_many_files: Too many files
   • file_too_large: File too large
   • document_indexing: Document indexing failed
   • not_found: Document not found
   • dataset_name_duplicate: Dataset name duplicate
   • dataset_not_initialized: Dataset not initialized ^[1][2].

2. Verify Celery Worker Service: Ensure that the Celery worker service is running correctly. This service is responsible for processing tasks such as document indexing. You can verify and troubleshoot the Celery worker service by following these steps:

   • Start the Celery worker service with the appropriate command for your system.
   • Check the logs for any errors or warnings.
   • Ensure all dependencies are correctly installed.
   • Verify that the .env file is correctly configured.
   • Ensure that the database is migrated to the latest version.
   • Verify that all required middleware services (PostgreSQL, Redis, Weaviate) are running ^[3][4].

3. Logs and Debugging Tools: Utilize Dify's Logs feature to diagnose issues with document uploads. This feature allows you to observe and mark the interactions of the Dify app, providing insights into user inputs, AI outputs, and evaluations by users and operators ^[5].

By following these steps, you should be able to identify and resolve the issue causing your files to be stuck in the "Queuing" status.

To continue talking to Dosu, mention @dosu.

[crazywoola]

We are currently working on it

[crazywoola]

It's resolved now.