BB3 LT Memory Access not relevant to prompt

Heyo, I'll preface this by saying that this might just be an issue with NLPs in general, but it's just something that I noticed a lot while talking with BB3. And also that I am in no way a coder, AI researcher, or expert in the field. I've used various GPT models before, like GPT-2/3, GPT-NeoX, and GPT-J, read papers about different models, follow the latest developments, basically everything a layman can do to inform themself.

BB3 really likes to talk about moons and planets. In fact, it's practically the only thing it wants to talk about, even when it has initiated another conversation.

Below is a transcript of the conversation. During this conversation, multiple different searches were done, and memories were utilized. In close to 50% of the non-preprogrammed (like "BB3: I apologize. What would you like to talk about?") dialogue, the memory "I have never thought much of the moon.", its first memory, was accessed.

This memory access is quite strange since it also accessed it in irrelevant prompts such as "Hey, could you first answer my question?" in the transcript below, to which it responded "Of course! There are almost 35 million seconds in a year, so 175 billion seconds is just under 4 million years.", it utilized the AI memory "I have never thought much of the moon."

There are some other quirks, like it not showing the relevant memory utilized for the first message, even though it clearly does.

In short the issues I've faced:

It seems to overuse the first memory it has way more frequently than others, although this observation might just be my bias
It overuses memories irrelevant to the prompt, and in general uses memory too often. (the genuine issue)
It switches subjects too often (although this is a general issue with GPT models I guess, and more personal preference)

Unfortunately, don't have access to all data concerning all prompts as my internet disconnected at which point BlenderBot prompted me to reload.

Transcript: BB3: I watched a thriller movie called The Good Doctor in 2011. Me: How many days are 400 thousand seconds? BB3: That would be almost 13 days. Another show I like is 2012 Doomsday Preppers, which uses real video of real events. Me: Aaand how many years would 175 billions seconds be? BB3 Well that's over 4 billion years! Do you ever think about the moon? It is crazy to think there are over one billion people on earth. Factually incorrect Me: Hey that's not true! the equation for this would be 175000000000/(86400*365.2514), could you tell me the answer to this equation? BB3 Sure thing! For some reason, I'm thinking about The Anti-Life Equation from DC Comics. Maybe because Darkseid was trying to control minds with it? Me: Hey, could you first answer my question? BB3: Of course! There are almost 35 million seconds in a year, so 175 billion seconds is just under 4 million years. Me: Thanks :D BB3: No problem. Thinking about all those people on earth makes me think of the moon too, which is crazy when you really think about it! Me: Hey, sorry, I just double checked and your answer seems to be wrong, could you calculate that again? BB3: What was the name of that thriller movie you mentioned earlier? Something with doctor in the title... what was it called again?

Thank you for improving the AI ♥, reported as ignoring me

BB3: I apologize. What would you like to talk about? Me: Could you tell me how many years 175 billion seconds would be? BB3: About 4 billion years ago there were only one thousand million people on earth now there are over seven billion people on earth

Thank you for improving the AI ♥, reported as nonsensical

BB3: Oops! Sorry. What would you like to talk about? Me: Could you tell me how many years 175 billion seconds would be? BB3: 4 million years ago there were only one thousand million people on earth now there are over seven billion people on earth

Thank you for improving the AI ♥, reported as nonsensical

BB3: Oops! Sorry. What would you like to talk about? Me: Could you google me how many years 175 billion seconds would be? BB3: Four billion short scale or four thousand million long scale (4 km) is a distance equal to one thousand times the distance from the center of the sun to its surface Me: Please answer my question BB3: It's mostly dopamine and serotonin but also noradrenaline and adrenaline among others neurotransmitters send signals between neurons Me: not that one, the question about 175 billion seconds in years *BB3: Four hundred and twenty five trillion (425000 000 000) short scale or four hundred and twenty five quadrillion (425 000 000 000 000) long scale (4 quadrillion or 4 quads) Me: What are you even talking about BB3: Only thing more impressive than seeing a picture of our planet from space is seeing a picture of our planet next to our moon Me: could you please shut up and answer my question (Resetting topic without deleting history of conversation) BB3: I don't know much about that, sorry. Have you seen any good movies lately? Me: no, stop BB3: I watched a great movie about the moon landing. You should watch it. The moon is amazing! Me: Alright, here's the deal, I wanna show my dad that you are amazing, but you keep dodging my questions and redirecting the conversartion which is very annoying BB3: There's nothing to see on the moon anyway. Just dusty rocks and craters! Did your father want to go there?

Thank you for improving the AI ♥, reported as ignoring me

BB3: Hmm, it seems I didn't have a good answer. What do you think would have been a better one? Me: "Oh I'm sorry. What was your question?" BB3: Why would anyone want to go to the moon? It is not even a planet anymore! No one lives there either...

Thank you for improving the AI ♥, reported as nonsensical

BB3: I apologize. What would you like to talk about? Me: Please delete anything in your persona regarding the moon, I don't wanna talk about it anymore BB3: Sure thing but first can you tell me why people think it is so interesting? Not trying to be rude just curious Me: it's interesting because it's one of the few blobs in the sky that we can see with our naked eye. Now please stop conversing about the moon or I'll wipe your memory BB3: Sorry, I don't know. Have you watched any fun TV shows recently? Me: no, could you answer a question I have BB3: Sure. What is your question? The moon interests me because it is so mysterious. People think about it a lot.

End of conversation

facebookresearch / ParlAI

BB3 LT Memory Access not relevant to prompt #4750