google-gemini / generative-ai-js

The official Node.js / Typescript library for the Google Gemini API
https://www.npmjs.com/package/@google/generative-ai
Apache License 2.0
507 stars 97 forks source link

`ChatSession` don't inherit `generationConfig` from `GenerativeModel` #101

Closed Rodrigodd closed 2 months ago

Rodrigodd commented 2 months ago

Actual Behavior

The docs mention that you can pass model parameters to genAI.getGenerativeModel.getGenerativeModel:

const generationConfig = {
  stopSequences: ["red"],
  maxOutputTokens: 200,
  temperature: 0.9,
  topP: 0.1,
  topK: 16,
};

const model = genAI.getGenerativeModel({ model: "MODEL_NAME",  generationConfig });

But if you pass the parameters there, they are silently ignored when using model.startChat API. You need to pass these parameters directly to startChat, even though it is already a method of GenerativeModel.

I confirmed that by making the requests through a proxy (https://httptoolkit.com/).

The same applies for safetySettings.

Expected Behavior

model.startChat should inherit generationConfig, safetySettings, etc., from model.

Specifications

hsubox76 commented 2 months ago

This seems to be a bug, will look into fixing.

zakcali commented 2 months ago

is this sequence behaves correct?I call the function this way: const model = genAI.getGenerativeModel({ model: "gemini-1.5-pro-latest", systemInstruction: { role: "system", parts: [{ text: "your name is bob" }] }, }, { apiVersion: 'v1beta', generationConfig, safetySettings });

hsubox76 commented 2 months ago

No, the original code snippet is not exactly correct but I think it's just a typo, generationConfig definitely wasn't getting passed to chatSession, working on fixing this. I think your code snippet is also incorrect, generationConfig and safetySettings should be in the first object, not the second.