ERROR: `restricted HarmBlockThreshold setting BLOCK_NONE` on Vertex AI - Gemini

totsukash commented 4 days ago

Self Checks

[X] This is only for bug report, if you would like to ask a question, please head to Discussions.
[X] I have searched for existing issues search for existing issues, including closed ones.
[X] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
[X] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
[X] Please do not modify this template :) and fill in all the required fields.

Dify version

v0.8.3

Cloud or Self Hosted

Cloud

Steps to reproduce

After completing the Vertex AI setup, an error occurs when calling the LLM.

Run failed: [vertex_ai] Bad Request Error, 400 User has requested a restricted HarmBlockThreshold setting BLOCK_NONE. You can get access either (a) through an allowlist via your Google account team, or (b) by switching your account type to monthly invoiced billing via this instruction: https://cloud.google.com/billing/docs/how-to/invoiced-billing.

I know the source of the problem.

https://github.com/langgenius/dify/blob/64baedb48429bf7332feab61d7a83cf1b7c91cda/api/core/model_runtime/model_providers/vertex_ai/llm/llm.py#L506-L511

✔️ Expected Behavior

To be able to perform inference without errors, just like other LLMs.

❌ Actual Behavior

Errors are occurring.

dosubot[bot] commented 4 days ago

Hey there, @totsukash! I'm here to help you with bugs, questions, and becoming a contributor. Let me know how I can assist you while you wait for a human maintainer.

To resolve the "Bad Request Error, 400" related to the restricted HarmBlockThreshold setting BLOCK_NONE when using Vertex AI with Dify, you need to adjust the HarmBlockThreshold settings in the safety_settings dictionary. The current settings are:

safety_settings = {
    HarmCategory.HARM_CATEGORY_HARASSMENT: HarmBlockThreshold.BLOCK_NONE,
    HarmCategory.HARM_CATEGORY_HATE_SPEECH: HarmBlockThreshold.BLOCK_NONE,
    HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT: HarmBlockThreshold.BLOCK_NONE,
    HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT: HarmBlockThreshold.BLOCK_NONE,
}

To resolve the error, you should change the HarmBlockThreshold values to a less restrictive setting. For example, you can use HarmBlockThreshold.BLOCK_LOW:

safety_settings = {
    HarmCategory.HARM_CATEGORY_HARASSMENT: HarmBlockThreshold.BLOCK_LOW,
    HarmCategory.HARM_CATEGORY_HATE_SPEECH: HarmBlockThreshold.BLOCK_LOW,
    HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT: HarmBlockThreshold.BLOCK_LOW,
    HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT: HarmBlockThreshold.BLOCK_LOW,
}

This adjustment should help in resolving the "Bad Request Error, 400" ^[1].

To continue talking to Dosu, mention @dosu.

totsukash commented 4 days ago

Since we can't change that in the Cloud version, I'm thinking of modifying the default settings. https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/configure-safety-filters

I will set HARM_BLOCK_THRESHOLD_UNSPECIFIED or remove these configs.

langgenius / dify