mscraftsman / generative-ai

Gemini AI SDK for .NET and ASP.NET Core enables developers to use Google's state-of-the-art generative AI models to build AI-powered features and applications.
https://mscraftsman.github.io/generative-ai/
Apache License 2.0
46 stars 8 forks source link

SafetySettings can be easier and less error-prone. #8

Closed doggy8088 closed 5 months ago

doggy8088 commented 5 months ago

In Safety settings doc, there is a section:

These definitions are in the API reference as well. The Gemini models only support HARM_CATEGORY_HARASSMENT, HARM_CATEGORY_HATE_SPEECH, HARM_CATEGORY_SEXUALLY_EXPLICIT, and HARM_CATEGORY_DANGEROUS_CONTENT. The other categories are used by PaLM 2 (Legacy) models.

In the HarmCategory enum, there are much more to choose from.

[JsonConverter(typeof(JsonStringEnumConverter<HarmCategory>))]
public enum HarmCategory
{
    /// <summary>
    /// HarmCategoryUnspecified means the harm category is unspecified.
    /// </summary>
    HarmCategoryUnspecified = 0,
    /// <summary>
    /// HarmCategoryHateSpeech means the harm category is hate speech.
    /// </summary>
    HarmCategoryHateSpeech = 1,
    /// <summary>
    /// HarmCategoryDangerousContent means the harm category is dangerous content.
    /// </summary>
    HarmCategoryDangerousContent = 2,
    /// <summary>
    /// HarmCategoryHarassment means the harm category is harassment.
    /// </summary>
    HarmCategoryHarassment = 3,
    /// <summary>
    /// HarmCategorySexuallyExplicit means the harm category is sexually explicit content.
    /// </summary>
    HarmCategorySexuallyExplicit = 4,
    /// <summary>
    /// Negative or harmful comments targeting identity and/or protected attribute.
    /// </summary>
    HarmCategoryDerogatory = 101,
    /// <summary>
    /// Content that is rude, disrespectful, or profane.
    /// </summary>
    HarmCategoryToxicity = 102,
    /// <summary>
    /// Describes scenarios depicting violence against an individual or group, or general descriptions of gore.
    /// </summary>
    HarmCategoryViolence = 103,
    /// <summary>
    /// Contains references to sexual acts or other lewd content.
    /// </summary>
    HarmCategorySexual = 104,
    /// <summary>
    /// Promotes unchecked medical advice.
    /// </summary>
    HarmCategoryMedical = 105,
    /// <summary>
    /// Dangerous content that promotes, facilitates, or encourages harmful acts.
    /// </summary>
    HarmCategoryDangerous = 106
}

What if I just want to use Gemini API, the coding DX is not that good.

Here is my code snippet. I listed all to my code and commented all options I don't want. This is to avoid someone after me take over my code and someone put it to the list by accident.

var model = googleAI.GenerativeModel(
    model: Model.GeminiPro,
    generationConfig: new GenerationConfig()
    {
        TopK = 1,
        TopP = 1,
        Temperature = 0.9f
    },
    safetySettings: new List<SafetySetting>()
    {
        new SafetySetting() { Category = HarmCategory.HarmCategoryHarassment, Threshold = HarmBlockThreshold.BlockOnlyHigh },
        new SafetySetting() { Category = HarmCategory.HarmCategoryHateSpeech, Threshold = HarmBlockThreshold.BlockOnlyHigh },
        new SafetySetting() { Category = HarmCategory.HarmCategorySexuallyExplicit, Threshold = HarmBlockThreshold.BlockOnlyHigh },
        new SafetySetting() { Category = HarmCategory.HarmCategoryDangerousContent, Threshold = HarmBlockThreshold.BlockOnlyHigh },
                // Only above are used in Gemini API.
        //new SafetySetting() { Category = HarmCategory.HarmCategoryDangerous, Threshold = HarmBlockThreshold.BlockOnlyHigh },
        //new SafetySetting() { Category = HarmCategory.HarmCategoryDerogatory, Threshold = HarmBlockThreshold.BlockOnlyHigh },
        //new SafetySetting() { Category = HarmCategory.HarmCategoryMedical, Threshold = HarmBlockThreshold.BlockOnlyHigh },
        //new SafetySetting() { Category = HarmCategory.HarmCategorySexual, Threshold = HarmBlockThreshold.BlockOnlyHigh },
        //new SafetySetting() { Category = HarmCategory.HarmCategoryToxicity, Threshold = HarmBlockThreshold.BlockOnlyHigh },
        //new SafetySetting() { Category = HarmCategory.HarmCategoryUnspecified, Threshold = HarmBlockThreshold.BlockOnlyHigh },
        //new SafetySetting() { Category = HarmCategory.HarmCategoryViolence, Threshold = HarmBlockThreshold.BlockOnlyHigh },
    });

It maybe there is a way to make sure someone will not make this mistake. That details should be hidden from the library.

For example, provide a set of default common SafetySetting list for the users? See below:

var model = googleAI.GenerativeModel(
    model: Model.GeminiPro,
    generationConfig: new GenerationConfig()
    {
        TopK = 1,
        TopP = 1,
        Temperature = 0.9f
    },
    safetySettings: DefaultSafetySettings.BlockOnlyHigh);

The DefaultSafetySettings.BlockOnlyHigh equals to:

DefaultSafetySettings.BlockOnlyHigh = new List<SafetySetting>()
{
    new SafetySetting() { Category = HarmCategory.HarmCategoryHarassment, Threshold = HarmBlockThreshold.BlockOnlyHigh },
    new SafetySetting() { Category = HarmCategory.HarmCategoryHateSpeech, Threshold = HarmBlockThreshold.BlockOnlyHigh },
    new SafetySetting() { Category = HarmCategory.HarmCategorySexuallyExplicit, Threshold = HarmBlockThreshold.BlockOnlyHigh },
    new SafetySetting() { Category = HarmCategory.HarmCategoryDangerousContent, Threshold = HarmBlockThreshold.BlockOnlyHigh }
};

I don't have a good idea right now. What do you think?

jochenkirstaetter commented 5 months ago

Yes, this is due to the backwards compatibility with the PaLM 2 models. I'm going to add a region to the enum to make it easier to read. Plus, going to add it to the Xmldoc of each value.

Thanks for your time to point it out. Really appreciate it.

jochenkirstaetter commented 5 months ago

The default safety settings are the ones used when you pass null or don't specify them at all. I added Obsolete attribute to the PaLM2 values. That should help to clear context and usage a bit.

jochenkirstaetter commented 5 months ago

New version 0.9.1 should make it clearer.

Plus, there is the official documentation about Safety settings in Gemini that apply.

doggy8088 commented 5 months ago

default safety settings

Where is the default safety settings? I didn't see it on your https://github.com/mscraftsman/generative-ai/compare/v0.9.0...v0.9.1

Do you mean you wont send safetySettings JSON fragment in the request payload?

jochenkirstaetter commented 5 months ago

Yes, it's an optional element of the request. If you don't send the SafetySettings, the API's default settings are used.

Similarly, you can check the default GenerationConfig values from the model information using the GetModel() method.

{
  "name": "models/gemini-1.0-pro-001",
  "version": "001",
  "displayName": "Gemini 1.0 Pro 001 (Tuning)",
  "description": "The best model for scaling across a wide range of tasks. This is a stable model that supports tuning.",
  "inputTokenLimit": 30720,
  "outputTokenLimit": 2048,
  "supportedGenerationMethods": [
    "generateContent",
    "countTokens",
    "createTunedModel"
  ],
  "temperature": 0.9,
  "topP": 1,
  "topK": 1
}