langchain-ai / langchain-google

MIT License
74 stars 78 forks source link

When using streaming the generation_config params are not sent #308

Closed BigicecreamTaken closed 1 week ago

BigicecreamTaken commented 1 week ago

I use the following code:

from langchain_google_genai import GoogleGenerativeAI

llm = GoogleGenerativeAI(
    model="gemini-pro", google_api_key="<MY_API_KEY>", temperature=0
)
response = llm.invoke(
    "Tell me a joke about a cat."
)

print(response)
print("-----------------------------------")

for chunk in llm.stream("Tell me a joke about a cat."):
    print(chunk, end="", flush=True)

the temperature is set to 0 therefore I expect to get the same response every time(and the same for invoke and for stream)

when running the code the result is:

Why did the cat run away from the tree?

Because it was afraid of its bark!
-----------------------------------
Why did the cat get a job as a chef?

Because it was a purrfect cook!

the first part(the one return from the invoke function) is always the same while the result from the stream is not consistent.

the expected result is:

Why did the cat run away from the tree?

Because it was afraid of its bark!
-----------------------------------
Why did the cat run away from the tree?

Because it was afraid of its bark!