langgenius / dify

Dify is an open-source LLM app development platform. Dify's intuitive interface combines AI workflow, RAG pipeline, agent capabilities, model management, observability features and more, letting you quickly go from prototype to production.
https://dify.ai
Other
46.76k stars 6.61k forks source link

Add Safety settings to vertex #8733

Closed nl97 closed 4 days ago

nl97 commented 4 days ago

Self Checks

1. Is this request related to a challenge you're experiencing? Tell me about your story.

It would be great if for the model provider "Vertex AI" from Google Cloud, the safet settings would be added as an option:

"safetySettings": [
        {
            "category": "HARM_CATEGORY_HATE_SPEECH",
            "threshold": "BLOCK_LOW_AND_ABOVE"
        },
        {
            "category": "HARM_CATEGORY_DANGEROUS_CONTENT",
            "threshold": "BLOCK_LOW_AND_ABOVE"
        },
        {
            "category": "HARM_CATEGORY_SEXUALLY_EXPLICIT",
            "threshold": "BLOCK_LOW_AND_ABOVE"
        },
        {
            "category": "HARM_CATEGORY_HARASSMENT",
            "threshold": "BLOCK_LOW_AND_ABOVE"
        }
    ]

The default right now is no blocking, resulting in most users not being able to use it:

Bad Request Error, 400 User has requested a restricted HarmBlockThreshold setting BLOCK_NONE. You can get access either (a) through an allowlist via your Google account team, or (b) by switching your account type to monthly invoiced billing via this instruction: https://cloud.google.com/billing/docs/how-to/invoiced-billing.

2. Additional context or comments

No response

3. Can you help us with this feature?

crazywoola commented 4 days ago

Please refer this #8721