search

google-gemini / generative-ai-dart

The official Dart library for the Google Gemini API
https://ai.google.dev/gemini-api/docs/get-started/tutorial?lang=dart
Apache License 2.0
556 stars 114 forks source link

Unable to set candidateCount #34

Closed alfredobs97 closed 2 months ago

alfredobs97 commented 8 months ago

Hi! First of all, it very useful for Flutter & Dart developers to have this package to use with Gemini so thank you for your time creating the package.

While I was testing this package with old project that used PaLM API using HTTP calls, I faced this issue when I want to have more than one output

FormatException (FormatException: Unhandled GenerateContentResponse format: {error: {code: 400, message: Only one candidate can be specified, status: INVALID_ARGUMENT}})

My model instance and config

final model = GenerativeModel(
      model: 'gemini-pro',
      apiKey: apiKey!,
      generationConfig: GenerationConfig(candidateCount: 3),
    );

And the code that I used to fetch the Gemini responses

final prompt = await _promptRepository.getPromptForGemini(parameters);

final response = await model.generateContent([Content.text(prompt)]);

I'm not sure if this is how I have to use the package, I didn't see any example with more than one output.

natebosch commented 7 months ago

The FormatException should be resolved by https://github.com/google/generative-ai-dart/pull/36

It won't result in no exception though - we'd throw a different one with the message "Only one candidate can be specified".

This looks like an intentional error from the backend, but I'm not sure where to get more information about it. @anderdobo do you know who might be able to answer questions about unexpected errors from the LLM?

huynguyennovem commented 5 months ago

I have a semi-similar error when specifying candidateCount in GenerationConfig. Here is the full stack trace:

Error: Only one candidate can be specified
dart-sdk/lib/_internal/js_dev_runtime/private/ddc_runtime/errors.dart 297:3  throw_
packages/google_generative_ai/src/api.dart 509:38                            <fn>
packages/google_generative_ai/src/api.dart 499:10                            parseGenerateContentResponse
packages/google_generative_ai/src/model.dart 187:12                          generateContent
dart-sdk/lib/_internal/js_dev_runtime/patch/async_patch.dart 45:50           <fn>
dart-sdk/lib/async/zone.dart 1661:54                                         runUnary
dart-sdk/lib/async/future_impl.dart 162:18                                   handleValue
dart-sdk/lib/async/future_impl.dart 838:44                                   handleValueCallback
dart-sdk/lib/async/future_impl.dart 867:13                                   _propagateToListeners
dart-sdk/lib/async/future_impl.dart 643:5                                    [_completeWithValue]
dart-sdk/lib/async/future_impl.dart 713:7                                    callback
dart-sdk/lib/async/schedule_microtask.dart 40:11                             _microtaskLoop
dart-sdk/lib/async/schedule_microtask.dart 49:5                              _startMicrotaskLoop
dart-sdk/lib/_internal/js_dev_runtime/patch/async_patch.dart 181:7           <fn>

Looks like it hits parseError and https://github.com/google-gemini/generative-ai-dart/pull/36 should fix it?

Using google_generative_ai: ^0.3.2.

devoncarew commented 5 months ago

To see if this is fixed in main, you can try appending:

dependency_overrides:
  kittens:
    git:
      url: https://github.com/google-gemini/generative-ai-dart.git
      path: pkgs/google_generative_ai

to your pubspec. If this does address the problem, can you report back on this issue? That'll let us know to accelerate publishing the latest.

huynguyennovem commented 5 months ago

@devoncarew Not sure what kittens is (typo?) but I imported google_generative_ai as below:

dependency_overrides:
  google_generative_ai:
    git:
      url: https://github.com/google-gemini/generative-ai-dart.git
      path: pkgs/google_generative_ai

Here is the fetched version on pubspec.lock:

  google_generative_ai:
    dependency: "direct main"
    description:
      path: "pkgs/google_generative_ai"
      ref: HEAD
      resolved-ref: eb5e39b2533f40511d517ce873464dc76290159b
      url: "https://github.com/google-gemini/generative-ai-dart.git"
    source: git
    version: "0.3.3-wip"

But the error persists:

Error: Only one candidate can be specified
dart-sdk/lib/_internal/js_dev_runtime/private/ddc_runtime/errors.dart 297:3  throw_
packages/google_generative_ai/src/api.dart 509:38                            <fn>
packages/google_generative_ai/src/api.dart 499:10                            parseGenerateContentResponse
packages/google_generative_ai/src/model.dart 188:12                          generateContent
dart-sdk/lib/_internal/js_dev_runtime/patch/async_patch.dart 45:50           <fn>
dart-sdk/lib/async/zone.dart 1661:54                                         runUnary
dart-sdk/lib/async/future_impl.dart 162:18                                   handleValue
dart-sdk/lib/async/future_impl.dart 838:44                                   handleValueCallback
dart-sdk/lib/async/future_impl.dart 867:13                                   _propagateToListeners
dart-sdk/lib/async/future_impl.dart 643:5                                    [_completeWithValue]
dart-sdk/lib/async/future_impl.dart 713:7                                    callback
dart-sdk/lib/async/schedule_microtask.dart 40:11                             _microtaskLoop
dart-sdk/lib/async/schedule_microtask.dart 49:5                              _startMicrotaskLoop
dart-sdk/lib/_internal/js_dev_runtime/patch/async_patch.dart 181:7           <fn>
devoncarew commented 5 months ago

@devoncarew Not sure what kittens is (typo?) but I imported google_generative_ai as below:

Oops! That was a copy-paste error; it should have read google_generative_ai :)

arnoorani commented 2 months ago

is this issue resolved I still get this issue, while using google_generative_ai: ^0.4.3

arnoorani commented 2 months ago

@devoncarew Not sure what kittens is (typo?) but I imported google_generative_ai as below:

dependency_overrides:
  google_generative_ai:
    git:
      url: https://github.com/google-gemini/generative-ai-dart.git
      path: pkgs/google_generative_ai

Here is the fetched version on pubspec.lock:

  google_generative_ai:
    dependency: "direct main"
    description:
      path: "pkgs/google_generative_ai"
      ref: HEAD
      resolved-ref: eb5e39b2533f40511d517ce873464dc76290159b
      url: "https://github.com/google-gemini/generative-ai-dart.git"
    source: git
    version: "0.3.3-wip"

But the error persists:

Error: Only one candidate can be specified
dart-sdk/lib/_internal/js_dev_runtime/private/ddc_runtime/errors.dart 297:3  throw_
packages/google_generative_ai/src/api.dart 509:38                            <fn>
packages/google_generative_ai/src/api.dart 499:10                            parseGenerateContentResponse
packages/google_generative_ai/src/model.dart 188:12                          generateContent
dart-sdk/lib/_internal/js_dev_runtime/patch/async_patch.dart 45:50           <fn>
dart-sdk/lib/async/zone.dart 1661:54                                         runUnary
dart-sdk/lib/async/future_impl.dart 162:18                                   handleValue
dart-sdk/lib/async/future_impl.dart 838:44                                   handleValueCallback
dart-sdk/lib/async/future_impl.dart 867:13                                   _propagateToListeners
dart-sdk/lib/async/future_impl.dart 643:5                                    [_completeWithValue]
dart-sdk/lib/async/future_impl.dart 713:7                                    callback
dart-sdk/lib/async/schedule_microtask.dart 40:11                             _microtaskLoop
dart-sdk/lib/async/schedule_microtask.dart 49:5                              _startMicrotaskLoop
dart-sdk/lib/_internal/js_dev_runtime/patch/async_patch.dart 181:7           <fn>

were you able to resolve the issue?

devoncarew commented 2 months ago

@arnoorani - given the age of this issue (and how much the library has changed since then), this may or may not be the same problem. Do you mind filing a new issue w/ what you're seeing, including a stack trace? Thanks!

natebosch commented 2 months ago

Note that this error:

Error: Only one candidate can be specified

Is not a bug with the SDK. This is a correctly reported error produced by the backend. You'll need to remove the candidateCount argument, or set it to 1 for this request. I don't know the full set of backend restrictions that limit the candidate count to one.

arnoorani commented 2 months ago

@arnoorani - given the age of this issue (and how much the library has changed since then), this may or may not be the same problem. Do you mind filing a new issue w/ what you're seeing, including a stack trace? Thanks!

The issue does not come up anymore, but will keep an eye out for it, thanks so much for the response.