search

keldenl / gpt-llama.cpp

A llama.cpp drop-in replacement for OpenAI's GPT endpoints, allowing GPT-powered apps to run off local llama.cpp models instead of OpenAI.
MIT License
594 stars 66 forks source link

Every Other Chat Response #56

Open msj121 opened 1 year ago

msj121 commented 1 year ago

I get a restart on the chatRoute because the last response is recorded in the global messages like this:

{"role":"assistant","content":"\\\b\\\b \b\nUSER:\\\b\\\b \b\ntests\nassistant:\nI'm happy to assist you in finding information related to tests. What specific topic or query are you interested in?"}

Only happens every other response, which might be the oddest piece.

It appears to me that it is doing something odd by combining the previous user response with the assistant response.... Did you run into this?

Using WizardLM

msj121 commented 1 year ago

I altered the following line: [line #195 of chatRoutes](https://github.com/keldenl/gpt-llama.cpp/blob/1c8b1c1ae85a80c343a8979046d95d0abc5ec377/routes/chatRoutes.js#L195)

to be:

if (!responseStart) {

Now it works properly.

What is that check for? Maybe it can be done in a more reliable way? Happy to make a PR but obviously that logic is for something.