SciSharp / LLamaSharp

A C#/.NET library to run LLM (🦙LLaMA/LLaVA) on your local device efficiently.
https://scisharp.github.io/LLamaSharp
MIT License
2.72k stars 352 forks source link

[BUG]: ChatSession unnecessarily prevents arbitrary conversation interleaving #857

Open lostmsu opened 4 months ago

lostmsu commented 4 months ago

Description

With asynchronous flows user might ask 2 questions in succession before model has time to answer the first one. This ends up in the following chat pattern:

user: multiply 2 by 3 user: then divide by -1 assistant: 6 assistant: -6

Unfortunately, for some reason ChatSession prevents this while LLAMA 3 instruct models handle such scenario without any issues.

Reproduction Steps

var history = new ChatHistory();
history.AddMessage(AuthorRole.User, "multiply 2 by 3");
history.AddMessage(AuthorRole.User, "then divide by -1");
history.AddMessage(AuthorRole.Assistant, "6");
history.AddMessage(AuthorRole.Assistant, "-6");

Throws ArgumentException twice

Environment & Configuration

Known Workarounds

No response