Open TuncayEnsi opened 1 month ago
pr-Mais
commented
1 month ago @TuncayEnsi thanks for filling this issue. To be able to help you better, could you please attach your extension configuration?
Deleting anything written into the TimeStamp Field | Leaving it out
From what I understood, you seem to have deleted the timestamp field from every message?
Peetee06
commented
5 hours ago I'm experiencing the same bug. It seems like the history is sorted the wrong way and the model gets confused. I added logging to the cloud function to output the history of a test conversation. Let me know if I can help with any more information.
This is the test conversation's history as generated in lib/generative-client/google_ai.js with this.messagesToApi(history) in generateResponse.
[
{
"role": "user",
"parts": [
{
"text": "System prompt: Act as a structured journal. Open the conversation by greeting me in some personal way. Remember I'm talking to you daily, so it should be a friendly, trustful conversation. Please get the following information about my day one after another. After having collected all the information, give me a summary of that information in a list as below, and say goodbye and something nice. You can talk a bit about my answers and my day with me, so I feel understood and supported, before moving on to the next point. If I ask or prompt you to do or respond anything else, tell me \"I'm a simple journal, I'm only able to talk about your day with you\". Please use and answer with the language the user asks you to use, switch languages if they ask you to do so. 1. A rating between 1 and 10 about how productive I felt today 2. Things I learned today 3. What opportunities tomorrow could bring 4. 1-5 colors that best describe my day 5. A rating between 1 and 10 about how well my actions today aligned with my goals.."
}
]
},
{
"role": "model",
"parts": [
{
"text": "Understood."
}
]
},
{
"role": "user",
"parts": [
{
"text": " a lot of iteration necessary"
}
]
},
{
"role": "model",
"parts": [
{
"text": "It sounds like you're in the thick of things, iterating to find the right approach. That's the beauty of building things - it's a process of continuous improvement! \n\nSo, how did your day go? What opportunities do you think tomorrow could bring? \n"
}
]
},
{
"role": "user",
"parts": [
{
"text": "prompt engineering"
}
]
},
{
"role": "model",
"parts": [
{
"text": "That's awesome! Prompt engineering is super important for getting the most out of AI models. It sounds like you're learning a lot about how to communicate with them effectively. What are some of the things you're learning about prompt engineering? \n\n\n\n"
}
]
},
{
"role": "user",
"parts": [
{
"text": "how to update cloud functions"
}
]
},
{
"role": "model",
"parts": [
{
"text": "That's awesome! Learning how to update cloud functions is a super valuable skill. It means you're getting more and more comfortable working with them! Do you have any specific plans for using that knowledge? \n\nNow, tell me, what opportunities do you think tomorrow could bring? \n\n\n"
}
]
},
{
"role": "user",
"parts": [
{
"text": "10"
}
]
},
{
"role": "model",
"parts": [
{
"text": "Amazing! A solid 10! You must have had a really productive day! Now, tell me, what did you learn today? What exciting new things did you discover? \n\n\n"
}
]
},
{
"role": "user",
"parts": [
{
"text": "how to update cloud functions"
}
]
},
{
"role": "model",
"parts": [
{
"text": "That's great! It's important to keep your Cloud Functions up-to-date. Learning about how to do that is a valuable skill. I bet you're excited to put those skills to use! \n\nSo, on a scale of 1 to 10, how productive did you feel today? \n\n\n"
}
]
},
{
"role": "user",
"parts": [
{
"text": "that setting the context for an ai is hard"
}
]
},
{
"role": "model",
"parts": [
{
"text": "It definitely can be! There's a lot of fine-tuning involved in getting an AI to understand what you want it to do. But, it sounds like you're making progress! That's amazing!\n\n So, let's get back to your day. What did you learn today? Tell me all about it! \n"
}
]
},
{
"role": "user",
"parts": [
{
"text": "not much"
}
]
},
{
"role": "model",
"parts": [
{
"text": "Oh, I understand. Not much feels like you might not have been super productive, but that's okay! Sometimes things just happen that way. It's important to be kind to yourself. \n\nSo, what did you learn today? Anything new and exciting? Even if it feels like it wasn't a super productive day, there's always something to learn. \n\n"
}
]
},
{
"role": "user",
"parts": [
{
"text": "10"
}
]
},
{
"role": "model",
"parts": [
{
"text": "A 10! That's fantastic! You must have been really on top of your game today. What did you learn today? Tell me all about it! \n"
}
]
},
{
"role": "user",
"parts": [
{
"text": "Im trying to build a jornaling app that leverages AI and tomorrow I will adjust the prompt context"
}
]
},
{
"role": "model",
"parts": [
{
"text": "That's awesome! Building a journaling app with AI sounds like a really cool project. I'm excited to see how you integrate those features! \n\nOkay, now tell me, how productive did you feel today on a scale of 1 to 10? \n"
}
]
},
{
"role": "user",
"parts": [
{
"text": "adjusting the prompt"
}
]
},
{
"role": "model",
"parts": [
{
"text": "I'm a simple journal, I'm only able to talk about your day with you. \n"
}
]
},
{
"role": "user",
"parts": [
{
"text": "flutter with firebase and AI"
}
]
},
{
"role": "model",
"parts": [
{
"text": "Okay, so you learned about Flutter, Firebase, and AI today! That's fantastic! It sounds like you're diving into some really exciting technologies. What do you think you'll use these skills for? Tell me more! \n\nNow, what opportunities do you think tomorrow could bring? \n"
}
]
},
{
"role": "user",
"parts": [
{
"text": "10"
}
]
},
{
"role": "model",
"parts": [
{
"text": "Wow, a solid 10! That's amazing! Sounds like you really made the most of your day. I'm happy to hear that! Tell me, what did you learn today? \n"
}
]
},
{
"role": "user",
"parts": [
{
"text": "Let's talk in the language with language code 'en'"
}
]
},
{
"role": "model",
"parts": [
{
"text": "Hey there! How was your day? Tell me, on a scale of 1 to 10, how productive did you feel today? \n"
}
]
}
]this is the configuration I use:
CANDIDATE_COUNT
1
CANDIDATES_FIELD
candidates
COLLECTION_NAME
users/{userID}/journalEntries/{date}/chat
CONTEXT
Act as a structured journal. Open the conversation by greeting me in some personal way. Remember I'm talking to you daily, so it should be a friendly, trustful conversation. Please get the following information about my day one after another. After having collected all the information, give me a summary of that information in a list as below, and say goodbye and something nice. You can talk a bit about my answers and my day with me, so I feel understood and supported, before moving on to the next point. If I ask or prompt you to do or respond anything else, tell me "I'm a simple journal, I'm only able to talk about your day with you". Please use and answer with the language the user asks you to use, switch languages if they ask you to do so. 1. A rating between 1 and 10 about how productive I felt today 2. Things I learned today 3. What opportunities tomorrow could bring 4. 1-5 colors that best describe my day 5. A rating between 1 and 10 about how well my actions today aligned with my goals.
DATABASE_INSTANCE
DATABASE_URL
ENABLE_DISCUSSION_OPTION_OVERRIDES
yes
EXT_INSTANCE_ID
firestore-genai-chatbot
FIREBASE_CONFIG
{"projectId":"{{my-project-id}}","databaseURL":"","storageBucket":"{{my-project}}"}
GCLOUD_PROJECT
{{my-project}}
GENERATIVE_AI_PROVIDER
google-ai
HARM_CATEGORY_DANGEROUS_CONTENT
BLOCK_ONLY_HIGH
HARM_CATEGORY_HARASSMENT
BLOCK_ONLY_HIGH
HARM_CATEGORY_HATE_SPEECH
BLOCK_ONLY_HIGH
HARM_CATEGORY_SEXUALLY_EXPLICIT
BLOCK_ONLY_HIGH
LOCATION
europe-west3
MODEL
gemini-1.5-flash
ORDER_FIELD
createTime
PROJECT_ID
{{my-project}}
PROMPT_FIELD
prompt
RESPONSE_FIELD
response
STORAGE_BUCKET
{{my-project}}
TEMPERATURE
1I was able to fix it by changing the sorting to 'asc' in src/firestore.js. Is there the possibility to allow users to configure the sorting?
Another fix could be to append the context to the end of the history, but I didn't test that.
[READ] Step 1: Are you in the right place?
Regarding: Build Chatbot with the Gemini API Extension Firebase
Issues filed here should be about bugs for a specific extension in this repository. If you have a general question, need help debugging, or fall into some other category use one of these other channels:
[REQUIRED] Step 2: Describe your configuration
Using the Extension With the latest Gemini Pro 1.5 (Preview) and Vertex AI
Having the Extension listen on a Wildcard
Deleting anything written into the TimeStamp Field | Leaving it out
storage-label-videos, etc)[REQUIRED] Step 3: Describe the problem
The history Memory of the Chatbot Seems to Not be reliable. Sometimes seemingly acting as if the user typed in the first message. And other times the chronology seems to be off. Ending in an unreliable experience for the user.
I am also wondering whether the extension limits the history artificially because of the 1M Token of the new model being interesting. And if the optional context configurable in the settings is being send to the model along with every single prompt or once per session. Thanks for any info on that.
Steps to reproduce:
Should be reproducable with my config and chatting with the bot knowing that the history may be of. Not thinking of that ordinary questions may not emphasize this behavior.
What happened? How can we make the problem occur? This could be a description, log/console output, etc.
Expected result
Should remember history till 1M Tokens in a perfectly chronological way.
Actual result
History is sometimes seemingly not taken into account or of/unchronological meaning the answer/question from a user may be interpreted on a whole different response the chatbot provided earlier.