googleapis / python-aiplatform

A Python SDK for Vertex AI, a fully managed, end-to-end platform for data science and machine learning.
Apache License 2.0
581 stars 319 forks source link

candidate_count = 2 for gemini-1.5-pro-preview-0409 produces an error, and/or the error message can be improved #3603

Open yarcyarc opened 2 months ago

yarcyarc commented 2 months ago

Hi,

If I understand it correctly, you're supposed to be able to set candidate_count to 2. I can't that to work. With candidate_count set to 1 the code works and a normal response is received.

Environment details

Ubuntu 22.04.4 LTS (Jammy Jellyfish)

Python 3.11.4

pip 24.0

Version: 1.47.0

Steps to reproduce

  1. Run the code below.
  2. Read exception

Code example

# loosely taken from https://github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_1_5_pro.ipynb
from vertexai.generative_models import GenerativeModel, GenerationConfig
gemini_pro_model = GenerativeModel("gemini-1.5-pro-preview-0409")
generation_config = GenerationConfig(
    temperature=2.0,
    candidate_count=2
)
model_response = gemini_pro_model.generate_content(contents="Why do cars have four wheels?", generation_config=generation_config)
print("model_response\n",model_response)

Stack trace

<..>
google.api_core.exceptions.InvalidArgument: 400 Unable to submit request because it has a candidateCount value of 2 but the supported range is from 1 to 2. Update the value and try again.
jtbg commented 1 month ago

Currently, the API only supports 1 candidate count for generative text

https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/gemini#parameters