The robot cannot read and retrieve the files inside the OSS file path

First-fan commented 1 week ago

Self Checks

[X] This is only for bug report, if you would like to ask a question, please head to Discussions.
[X] I have searched for existing issues search for existing issues, including closed ones.
[X] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
[X] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
[X] Please do not modify this template :) and fill in all the required fields.

Dify version

0.8.0

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

I gave the bot the OSS path to my file, but the bot told me it couldn't read it

✔️ Expected Behavior

No response

❌ Actual Behavior

No response

dosubot[bot] commented 1 week ago

Hello @First-fan! I'm here to help with any bugs, questions, or contributions you have for our project. Let me know how I can assist you!

The robot cannot read and retrieve the files inside the OSS file path in the self-hosted Dify version 0.8.0 because the file might be missing or the deployment method might have changed, leading to the deletion of the api/storage/privkeys directory. This directory is crucial as it contains the files used to encrypt the large model keys.

Additionally, if you are using a storage type other than local (e.g., S3, Azure Blob, or Aliyun OSS), ensure that the storage configuration is correctly set up. For example, if using Aliyun OSS, you need to configure the following environment variables:

ALIYUN_OSS_BUCKET_NAME
ALIYUN_OSS_ACCESS_KEY
ALIYUN_OSS_SECRET_KEY
ALIYUN_OSS_ENDPOINT
ALIYUN_OSS_REGION
ALIYUN_OSS_AUTH_VERSION
ALIYUN_OSS_PATH

To resolve the issue, you can reset the encryption key pair using the following commands:

For Docker Compose deployment:

docker exec -it docker-api-1 flask reset-encrypt-key-pair

For source code deployment, navigate to the api directory and run:

flask reset-encrypt-key-pair

Follow the prompts to complete the reset. This will regenerate the necessary encryption keys and restore the robot's ability to read and retrieve files from the OSS file path. Additionally, ensure that the STORAGE_LOCAL_PATH is correctly mounted if using local storage, especially when deploying with Docker or Docker Compose. If you encounter a "File not found" error, it may be due to switching deployment methods or deleting the api/storage/privkeys file ^[1]^[2]^[3]^[4].

To continue talking to Dosu, mention @dosu.

First-fan commented 1 week ago

@dosu Can our bots and large models of workflow calls read and retrieve the contents of the PDF file of the remote OSS

dosubot[bot] commented 1 week ago

Yes, your bots and large models of workflow calls can read and retrieve the contents of the PDF file from the remote OSS. Dify has ETL capabilities that include automated cleaning for various formats, including PDF ^[1].

To continue talking to Dosu, mention @dosu.

langgenius / dify