BadRequest: 400 POST https://generativelanguage.googleapis.com/v1beta/models/gemini-pro:generateContent?%24alt=json%3Benum-encoding%3Dint: Only one candidate can be specified

[Snigdha8]

I want to obtain all the possible candidates from the response of Gemini 1.0. In the code, when candidate_count = 1, it returns some response with 1 candidate. But when candidate_count > 1, then it returns the below error -

WARNING:tornado.access:400 POST /v1beta/models/gemini-pro:generateContent?%24alt=json%3Benum-encoding%3Dint (127.0.0.1) 795.11ms
---------------------------------------------------------------------------
BadRequest                                Traceback (most recent call last)
[<ipython-input-11-b57aabb4774d>](https://localhost:8080/#) in <cell line: 2>()
      1 question_1 = "What is Cucumber?"
----> 2 response = get_response(question_1)
      3 print("\n\nGemini response \n", response)

8 frames
[/usr/local/lib/python3.10/dist-packages/google/ai/generativelanguage_v1beta/services/generative_service/transports/rest.py](https://localhost:8080/#) in __call__(self, request, retry, timeout, metadata)
    844             # subclass.
    845             if response.status_code >= 400:
--> 846                 raise core_exceptions.from_http_response(response)
    847 
    848             # Return the response

BadRequest: 400 POST https://generativelanguage.googleapis.com/v1beta/models/gemini-pro:generateContent?%24alt=json%3Benum-encoding%3Dint: Only one candidate can be specified

Below is my code -

import google.generativeai as genai
import os
import PIL.Image

def get_response(question):

    safety_settings = [
        {
            "category": "HARM_CATEGORY_DANGEROUS",
            "threshold": "BLOCK_NONE",
        },
        {
            "category": "HARM_CATEGORY_HARASSMENT",
            "threshold": "BLOCK_NONE",
        },
        {
            "category": "HARM_CATEGORY_HATE_SPEECH",
            "threshold": "BLOCK_NONE",
        },
        {
            "category": "HARM_CATEGORY_SEXUALLY_EXPLICIT",
            "threshold": "BLOCK_NONE",
        },
        {
            "category": "HARM_CATEGORY_DANGEROUS_CONTENT",
            "threshold": "BLOCK_NONE",
        },
    ]

    config = genai.GenerationConfig(
      candidate_count=1,
      temperature=0.1
    )

    genai.configure(api_key=API_KEY)
    model = genai.GenerativeModel('gemini-pro', safety_settings=safety_settings, generation_config=config)
    response = model.generate_content(question)

    candidates = response.candidates  # Get all candidates
    print("\n")
    print("Length of candidates -> ", len(candidates))
    print("All candidates -> ", candidates)
    print("\n")
    candidates = response.candidates  # Get all candidates
    all_answers = [candidate.content for candidate in candidates]
    print("All answers -> ", all_answers)
    return response.text

question_1 = "What is Cucumber?"
response = get_response(question_1)
print("\n\nGemini response \n", response)

In general I have noticed for all the responses from Gemini, by default only 1 response is returned.

I am following the below documentation - https://ai.google.dev/api/python/google/generativeai/GenerationConfig https://ai.google.dev/api/rest/v1/GenerateContentResponse

Can someone please tell why candidate_count is not accepting values greater than 1, when the documentation mentions it as a configurable parameter? If there is another way of getting all responses from LLM, please share.

[Ark-kun]

The model you're using does not support returning multiple candidates (as the error message indicates). I'm not sure any publicly released model currently supports multiple candidates. The support for multiple candidates might be added in the future.

If there is another way of getting all responses from LLM, please share.

To get multiple responses, you can call generate_content multiple times.