search

nomic-ai / gpt4all

GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.
https://nomic.ai/gpt4all
MIT License
70.83k stars 7.71k forks source link

Change gpt4all API Server to remember context #2470

Open alijundi opened 5 months ago

alijundi commented 5 months ago

Hi guys,

I have been testing gpt4all using the assistant on windows. It is working fine. However, when I call the same sequence

role user role assistant role user role assistant role use

above sequence for example it seems that the API reads all like if the user sent all in one message and the response is not the same when using the client on windows!

I can see all messages appear in the server chat as if it all came in one message and this is not following the conversation sequence (user then assistance then user.....)

I double checked the json file going to the API and the roles are in sequence and correct.

Is there a bug here or what shall I do? I need to use gpt4al via the API

alijundi commented 5 months ago

I noticed that we don't remember the context in the APi server which renders working via APi useless

```

// don't remember any context resetContext();

ThiloteE commented 1 week ago

Related or duplicate of https://github.com/nomic-ai/gpt4all/issues/2602. There are two URLs (paths): /v1/completions and /v1/chat/completions. The former is for one-offs. Which one are you using? See also https://github.com/nomic-ai/gpt4all/wiki/Local-API-Server