Web server not working

[caioxapelao]

Bug Report

GPT4All Local server not working

Steps to Reproduce

1. Activate "Enable Local Server" Check Box

Expected Behavior

Local OpenAI API Endpoint

Your Environment

• GPT4All version: v3.0.0
• Operating System: Windows
• Chat model used : Any model

[FeliciaFiction]

OK, so I had an issue releated to the API yesterday and I found 1 thing, if you don't include the POST method you get an error 404. Since you don't include a log, detailed description of the error and the endpoint used this may or may not be related.

URI = http://x.x.x.x:4891/v1/completions Request body: "model" = "text-davinci-003" (filename of your model minus the extension so bla-model.gguf becomes bla-model) "prompt" = $query "max_tokens" = 500 "temperature" = 0.5

Request headers: Method = Post Contenttype = application/json (not sure if this is re quired)

[cosmic-snow]

I can assure you it is working. So I'm going to turn this into a question instead. See also:

• Settings documentation which has a short description for 'Enable Local Server' and 'API Server Port'

• 2689

Note: it is only accessible from localhost aka 127.0.0.1 and not through https, only http.

The easiest way to get a response is querying the list of models: http://localhost:4891/v1/models. This is a GET request, so you can easily do it in your browser.

Afterwards, you're most likely interested in talking to /v1/chat/completions, which is a POST request. Note that you need to somewhat follow the OpenAI API spec when querying the server. It's not a complete implementation, however.

As you're on Windows, here's an example with PowerShell using the model Phi-3 Mini Instruct:

Invoke-WebRequest -URI http://localhost:4891/v1/chat/completions -Method POST -ContentType application/json -Body '{
  "model": "Phi-3 Mini Instruct",
  "messages": [{"role":"user","content":"hi, who are you?"}],
  "max_tokens": 2048,
  "temperature": 0.7
}'

For others, here's the same example with cURL instead:

curl -X POST http://localhost:4891/v1/chat/completions -d '{
  "model": "Phi-3 Mini Instruct",
  "messages": [{"role":"user","content":"hi, who are you?"}],
  "max_tokens": 2048,
  "temperature": 0.7
}'

This should be clear enough for you to get it to work, right?

[cebtenzzre]

This issue is incomplete. Please provide additional information. What did you try, what did you expect to happen, and what actually happened? There are a few pitfalls to the local server that are not obvious from within the chat application, and we could probably document them better.

[cebtenzzre]

Closing due to lack of response, feel free to reopen if you have anything to add.