google / generative-ai-go

Go SDK for Google Generative AI
Apache License 2.0
487 stars 47 forks source link

Issue with `SetCandidateCount` Method - Only Receiving One Candidate #129

Open kaptinlin opened 1 month ago

kaptinlin commented 1 month ago

Description of the bug:

When using the SetCandidateCount method to set the number of candidates to 2 for the gemini-1.5-flash-latest model, the response still only returns 1 candidate. This issue occurs consistently and is not aligned with the expected behavior of the API.

Steps to reproduce:

  1. Create a client using the genai.NewClient method.
  2. Set the candidate count to 2 using model.SetCandidateCount(2).
  3. Start a chat session and send a message to the model.
  4. Observe the number of candidates in the response.

Code snippet:

package main

import (
    "context"
    "fmt"
    "log"
    "os"

    "github.com/joho/godotenv"
    "google.golang.org/api/option"
    "github.com/google/generative-ai-go/genai"
)

const (
    defaultModel  = "gemini-1.5-flash-latest"
    numCandidates = 2
)

func main() {
    // Load environment variables from the .env file.
    err := godotenv.Load()
    if err != nil {
        log.Fatalf("Error loading .env file: %v", err)
    }

    // Get the API key from the environment variable.
    apiKey := os.Getenv("GOOGLE_API_KEY")
    if apiKey == "" {
        log.Fatal("GOOGLE_API_KEY environment variable is not set")
    }

    ctx := context.Background()
    client, err := genai.NewClient(ctx, option.WithAPIKey(apiKey))
    if err != nil {
        log.Fatalf("Failed to create client: %v", err)
    }
    defer client.Close()

    model := client.GenerativeModel(defaultModel)
    model.SetTemperature(0)
    model.SetCandidateCount(numCandidates) 

    // Log the model settings
    fmt.Printf("Model: %s\nTemperature: %.1f\nCandidates: %d\n", defaultModel, 0.0, numCandidates)

    // Create a chat session.
    session := model.StartChat()

    // Send a message and get a response.
    msg := "Tell me a joke."
    resp, err := session.SendMessage(ctx, genai.Text(msg))
    if err != nil {
        log.Fatalf("Failed to send message: %v", err)
    }

    // Log the number of candidates received
    fmt.Printf("Received %d candidates\n", len(resp.Candidates))

    // Ensure we have the correct number of candidates.
    if len(resp.Candidates) != numCandidates {
        log.Fatalf("Expected %d candidates, got %d", numCandidates, len(resp.Candidates))
    }

    // Print the responses.
    for i, candidate := range resp.Candidates {
        fmt.Printf("Candidate %d: %s\n", i+1, candidate.Content)
    }
}

Actual vs expected behavior:

Expected: The response should contain 2 candidates as specified by SetCandidateCount. Actual: The response only contains 1 candidate.

Any other information you'd like to share?

The issue occurs consistently, regardless of the input message or context. This behavior may indicate a potential bug in the SetCandidateCount method or an API configuration issue.

eliben commented 2 weeks ago

This is the behavior of the underlying service right now, and we don't want to add SDK-specific error checks. Right now, models return a single candidate.

If you're interested, you can ask SDK-agnostic questions on the forum at https://discuss.ai.google.dev/