search

edgar971 / open-chat

A self-hosted, offline, ChatGPT-like chatbot with different LLM support. 100% private, with no data leaving your device.
MIT License
64 stars 8 forks source link

Error: connect ECONNREFUSED ::1:8000 #1

Open Joly0 opened 10 months ago

Joly0 commented 10 months ago

Hey, i am trying to setup Open-Chat as a docker in Unraid and when starting, i get this error in the docker logs image

though trying to open that http address through a browser (changing localhost to an actual ip of the host server) i am able to retrieve a correct response

ktfcaptain commented 10 months ago

Hey, i am trying to setup Open-Chat as a docker in Unraid and when starting, i get this error in the docker logs image

though trying to open that http address through a browser (changing localhost to an actual ip of the host server) i am able to retrieve a correct response

Ever figure anything out? I'm experiencing the same issue and was hoping someone would respond by now...

Joly0 commented 10 months ago

Nope, was not able to solve this yet. I have no idea why this is happening

edgar971 commented 10 months ago

Hey, i am trying to setup Open-Chat as a docker in Unraid and when starting, i get this error in the docker logs image

though trying to open that http address through a browser (changing localhost to an actual ip of the host server) i am able to retrieve a correct response

My assumption is that the API crashed. Can you share more details on your setup? Are you using NVIDIA GPU and do you have the drivers? Any other logs?

gellote commented 10 months ago

same problem

CodingAna commented 10 months ago

Bumping this, I have the same error.

Platform: Windows 10 (VM in unRAID) GPU: RTX 3070 (8GB VRAM) (Passthrough from unRAID), Driver version: 537.13 RAM: 20GB Model: 7B, 10 Layers loaded to the GPU (barely 2GB VRAM iirc)

My full log: https://pastebin.com/9qqP5jdb After some time of doing nothing I also get an out of memory error, even though my RAM and VRAM usage seems fine. Here's the additional log including the memory error: https://pastebin.com/U4uEXiVU

I'm using, except for the gpu layer count = 10, the default config, whatsoever when following the README.

Edit: For whatever reason it works now. I didn't do anything and when I was about to close the browser, I saw the chat window and was able to select the model. The chat also works. I don't know how and why, but it works now. Edit 2: Must have been the wind. When restarting the container, it throws the ECONNREFUSED error when loading the models. Edit 3:Okay, I think I know how to reproduce it at least. Loading the models will fail with the error until I get the out of memory error (which doesn't seem to have an effect to (V)RAM) and then the model data loads and I can use the chat.

gellote commented 10 months ago

In my case I solved it by loading version 7B instead of 13B. Since I have 8GB (GTX 1080) of vram and apparently the 13B model exceeds the memory.

gellote commented 10 months ago

image In model download url: https://huggingface.co/TheBloke/Nous-Hermes-Llama-2-7B-GGML/resolve/main/nous-hermes-llama-2-7b.ggmlv3.q4_0.bin

mrwhistles commented 8 months ago

Like most others my system was resolving localhost to IPv6 ::1. Setting environment variable "OPENAI_API_HOST=http://127.0.0.1:8000" solved the issue for me.