search

GoogleCloudPlatform / generative-ai

Sample code and notebooks for Generative AI on Google Cloud, with Gemini on Vertex AI
https://cloud.google.com/vertex-ai/docs/generative-ai/learn/overview
Apache License 2.0
6.62k stars 1.76k forks source link

The response text output from gemini-pro model have strange behavior, sometime no space in every words. (Vậythìchiềunayanhmuathịtbachỉvề) #345

Closed johnnietien closed 7 months ago

johnnietien commented 7 months ago

I recently got this issued when testing any kind of gemini model. I 'm now using Maker Suite API. And in Google AI Studio from Maker Suite some time I also see this problem but it not so often as the chatbot I was build from Streamlit. I 'm using prompt by English or Vietnamese language also get this issued. I did try to change all the value of Temperature, Top K, Top P also cannot fix the problem. I build an chatbot on Streamlit and use both google-generativeai library or either use langchain_google_genai also meet this problem. It doesn't happen since last time I use PALM2 chat model or text-bison.

Anyone has any idea about this or meet the same issued like this? If you have any solution how to fix this output please help me out. I did try regex or NLP library cannot fix this output and add some space between every words. Very appreciated if anyone can help. Thanks

Example of response error:

Khôngsaođâuạ,anhcóthểhỏiemvềbấtcứđiềugìanhmuốnbiết.Emluônởđâysẵn sàngtrảlờimọicâuhỏicủaanh.Anhcóthểhỏiemvềcuộcsốngcủaem,vềcôngviệccủaem,về sởthíchcủaem,hoặcbấtcứđiềugìkhácmàanhmuốnbiết.Emđềusẽtrảlờianhmộtcáchtrungthựcvàchi tiếtnhấtcóthể.
holtskinner commented 7 months ago

Hello, thank you for reporting this issue. However, this is not the appropriate repository for the Google AI/Maker Suite/Google AI Studio tools.

This repository is primarily focused on Generative AI for Google Cloud, not for Google AI.

I'd recommend reporting this in one of the Google AI repositories or Public Issue Trackers such as https://github.com/google/generative-ai-python

meystari commented 6 months ago

I just want to report here as this page pops up in google search when I look for rthis issue. I see a bug in python API repo for this https://github.com/google/generative-ai-python/issues/166

It would be great if google cloud platform can provide any help regarding this or escalate this issue, as this is only problem when using Google Cloud Function (especially Firebase Functions) .

These are the steps I have tried

  1. Changed temperature, top k, top p
  2. Tried different python version (unfortunately firebase functions puython only support python 310 and python 311)
  3. Asking AI to generate with spaces/human readable everytime.
  4. Tried using streaming version of API.
  5. Unfortunately only text version supported by this API sees to be gemini-pro, as I tried bison models it said its not supported. .