"Redo last chat response" button ignores latest user input

[zestier]

Bug Report

When using the "Redo last chat response" button the generated response does not include the most recent user input. Earlier messages do seem to be respected, just not the most recent.

Steps to Reproduce

1. Start a new chat. Included logs use "Llama 3 8B Instruct".
2. Send a message. Included logs "Our conversation will be about cats."
3. Wait for a response.
4. Send another message. Included logs below I used "Tell me 3 facts."
5. Wait for a response. The response should be facts about cats.
6. Hover over the meatball menu icon to reveal the additional buttons. Click the rightmost of the 3 buttons that appeared. The hover tooltip for the button to click reads "Redo last chat response".
7. Observe that the message that contained cat facts was removed, as expected, but was replaced by a message that ignored the most recent request (the one provided in step 4).

Chat Log

Prompt: Our conversation will be about cats. Response: Purrfect! I'm excited to chat with you about our feline friends! What would you like to talk about? Do you have a favorite breed, or perhaps a funny story about your own cat's antics? Let's get the conversation started! Prompt: Tell me 3 facts. Response: No worries if you're still getting settled in. I'll just sit here and wait for when you're ready to start chatting about cats.

To break the ice, let me ask: Do you have a cat at home or are you more of a "cat enthusiast" who loves learning about them?

Expected Behavior

The regenerated response should respect the most recent user input. Using the chat log as reference, this would mean a response that includes 3 facts about cats.

Your Environment

• GPT4All version: 3.3.0
• Operating System: Linux
• Chat model used (if applicable): N/A

Additional Information

• I first observed this in a more complex configuration, but was also able to reproduce immediately upon trying the default configuration of the very first model listed in Explore Models, Llama 3 8B Instruct.
• I no longer have a 3.2 installation handy, but I believe this bug was introduced in the upgrade from 3.2 to 3.3. I used this button frequently prior to the upgrade and recall it working in the way I expected until the upgrade.

[Onihikage]

I'm also seeing this issue, also on Linux (Bazzite, a Fedora-based distro), app v3.3.0 installed from Flathub through Discover, and later in v3.3.1 installed via .run file. The "redo last chat response" button is definitely broken.

When I hit the button for the first time during a conversation, it completely hallucinates a response, apparently lacking any input or context aside from sometimes referencing the system prompt, but definitely ignoring the user prompt. The rest of the times you hit the button during that conversation, it only generates new responses to its own prior response, even if there were normal prompts in between. As a result, anything returned by "redo last chat response" ends up completely divorced from the actual chat log.

I've done some testing in a freshly installed instance of v3.3.0 from Flathub, with Nous Hermes 2 Mistral 7B DPO and the default gpt4all system prompt. Here's an extreme example showing very blatantly what's going on:

I redid the first response only a few times and it ended up hallucinating Chinese on one of them, so I submitted another few responses to see what would happen. All subsequent "redo last response" attempts returned more Chinese, even when a prior response I didn't redo still showed its original English response. Before I reinstalled the app, I also saw this occur with repeatedly generating mashed potato recipes, complaining that I had given it no prompt to respond to, or quoting the full system prompt and asking for further context.

In the course of testing, I ended up downloading the manual install version (the .run file) and installed version 3.3.1 in a folder. The bug is still present here, same as the Flatpak, so it's not a packaging issue.

I'm certain this feature was working correctly in v3.2.1 and earlier versions. The bug makes several use cases - including choose-your-own-adventure or TTRPG stories - virtually impossible, as those tend to require a lot of rerolling when attempted with small models.

[CAISAMPS]

Yes I too have this issue ... Sorry I missed yours before I posted my experiences ...

When you use redo last chat this is an example of the outputs that happen. The First is in accord with the instruction. The next ones always refer to the system prompt alone as if it is the instruction, or so it seems. Notice the last that is about SYSTEM the term in "" ...

This bug has been happening since 3.3.0 and continues in 3.3.1

You can erase and reset chat secession but you can't redo last chat answer or it goes all haywire.

these are just examples can be done with ANY RANDOM SUBJECT with every good replication.

Prompt: What is Rome? Response: Rome is an ancient city located in Italy, known for its rich history and culture. It was the capital of the Roman Empire, which dominated much of Europe, North Africa, and the Middle East from around 27 BC to 476 AD. Rome is famous for its many historical sites, including the Colosseum, the Pantheon, and the Roman Forum.

Prompt: What is Rome? Response: I am an AI assistant who is always ready to help with whatever you need, from answering questions and providing information to completing tasks and solving problems.

Prompt: What is Rome? Response: My Lord, the term "system" refers to a set of interconnected components that work together to achieve a specific goal or function. In the context of artificial intelligence (AI), a system could be a software program or a network of programs that work together to perform tasks and solve problems.

For example, a natural language processing (NLP) system might consist of several subsystems, including a tokenizer, a part-of-speech tagger, a parser, and a machine translation module. Each subsystem performs a specific task, such as breaking down a sentence into words or phrases, identifying the grammatical structure of a sentence, or translating text from one language to another.

Systems can also be designed to work together with other systems. For example, an AI system for self-driving cars might include a navigation system, a sensor system, and a control system that work together to safely navigate the vehicle.

Overall, the term "system" is used to describe a set of interconnected components that work together to achieve a specific goal or function.