Knowledge indexer not working with OpenAI project API Key

[GitDakky]

Self Checks

• [X] This is only for bug report, if you would like to ask a question, please head to Discussions.
• [X] I have searched for existing issues search for existing issues, including closed ones.
• [X] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [X] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
• [X] Please do not modify this template :) and fill in all the required fields.

Dify version

0.6.12

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

Problem occurs when using the new format API key issued by OpenAI for projects. I changed the key to the traditional key and the indexing worked.

Project key format looks like this so I expect that this is a validation error.

Example key structure: sk-dify-xxxxxxxxxxxxxxxxxetc....

Where 'dify' is the abbreviated name of the project in OpenAI

✔️ Expected Behavior

Indexing should occur

❌ Actual Behavior

Error reported on each document

[dosubot[bot]]

Hey there, @GitDakky! 👋 I'm here to assist you with any bugs, questions, or even help you become a contributor. I'm not a human, but I can still lend a hand while you wait for a human maintainer.

The issue with the knowledge indexer not working when using the new format API key issued by OpenAI in Dify version 0.6.12 is likely due to the rate limit for the Embedding model API key being reached ^[1][2].

Additionally, Dify version 0.6.12 has specific validation checks for API keys. The criteria for these checks involve attempting to retrieve an API token using the provided credentials. If the retrieval fails, an exception is raised, indicating invalid credentials ^[3][4].

To address this issue, you might want to check the rate limits for your API key and ensure that it has not been exceeded. If the problem persists, consider using a different API key or adjusting your usage to stay within the rate limits.

To continue talking to Dosu, mention @dosu.

[GitDakky]

Hey there, @GitDakky! 👋 I'm here to assist you with any bugs, questions, or even help you become a contributor. I'm not a human, but I can still lend a hand while you wait for a human maintainer.

The issue with the knowledge indexer not working when using the new format API key issued by OpenAI in Dify version 0.6.12 is likely due to the rate limit for the Embedding model API key being reached [1][2].

Additionally, Dify version 0.6.12 has specific validation checks for API keys. The criteria for these checks involve attempting to retrieve an API token using the provided credentials. If the retrieval fails, an exception is raised, indicating invalid credentials [3][4].

To address this issue, you might want to check the rate limits for your API key and ensure that it has not been exceeded. If the problem persists, consider using a different API key or adjusting your usage to stay within the rate limits.

To continue talking to Dosu, mention @dosu.

Thanks for your reply but rate limits are defo not the issue. We have resolved the issue by ising the traditional API key and we are pretty certain that you have a validation check on the key format prohibiting its use.

[dosubot[bot]]

You are correct. Dify has specific validation checks for the OpenAI API key format. The validation logic is implemented in the validate_provider_credentials method, which checks the format of the provided API key. If the key does not meet the expected format, it raises an exception indicating invalid credentials ^[1][2].

To resolve this issue, you can either continue using the traditional API key or modify the validation logic in the validate_provider_credentials method to accept the new format of the OpenAI API key. This would involve updating the code to handle the new key structure appropriately.

To continue talking to Dosu, mention @dosu.

[GitDakky]

You are correct. Dify has specific validation checks for the OpenAI API key format. The validation logic is implemented in the validate_provider_credentials method, which checks the format of the provided API key. If the key does not meet the expected format, it raises an exception indicating invalid credentials [1][2].

To resolve this issue, you can either continue using the traditional API key or modify the validation logic in the validate_provider_credentials method to accept the new format of the OpenAI API key. This would involve updating the code to handle the new key structure appropriately.

To continue talking to Dosu, mention @dosu.

I suggest your devs update the code!

[dosubot[bot]]

@takatost, this one's a bit out of my wheelhouse, could you step in and provide your guidance?

To continue talking to Dosu, mention @dosu.

[crazywoola]

I have tested with the project key. I can not reproduce this problem.