🐛 [Bug]: Serge keeps babbling and doesn't stop

SparkieShock commented 2 months ago

Bug description

I have tried this for the first time so perhaps somewhere along the way I've missed a step. Though getting Serge and Docker installed onto Windows 11 was a slog to say the least for a non programmer. However .. I did manage to get it up and running and to download a model.

Upon entering the text 'Hello Computer' .. expecting it to just respond in kind ... it responds with an endless iteration of dialogue (that I have no way to stop incidentally). such as:

"Hello, I'm a computer. I have been helping people in classrooms for 3 years. I help students with their reports. I have been able to help people with their work. I am a computer. I work with students ......." And so on and on.

It seems I have a newborn child of an AI that doesn't actually know anything about where it is, what it is .. or what it's meant to do. The program is working .. but as you can tell from the response it gave it clearly is missing something. (HAL'S memory chips don't quite seem to be screwed in yet).

And what's more I couldn't stop this dialogue. so ultimately I had to crash the program to stop it.

I believe this was the Alpaca model (it was late last night and I was just trying to get it to work and after this failure I just turned off the PC with the thought to try again in the morning). The model was around 4 gigs in size and I believe was a 7B

Anyways .. if anyone has a clue about what is up with my attempt to create an AI on my machine I'd appreciate some insight. I know a little about computers but I'm not a programmer, I'm very much an end-level user. I've enjoyed using Chat GPT and wouldn't mind having this on my machine.

Steps to reproduce

In order to reproduce this I'm not sure .. this may be a problem specific to my machine or how I installed it. I followed the installation instructions from: https://www.howtogeek.com/881317/how-to-run-a-chatgpt-like-ai-on-your-own-pc/ Which is, incidentally, out of date as the link to the Serge Chat is now wrong. Also in several steps I had to do a lot of figuring out as to how I even get the Chat installed.

One aspect I wasn't able to solve was the command where one does: cp .env.sample .env

Because it kept returning an error to say this wasn't a valid command. And I couldn't even find a .env file within the 'serge' folder .. at least not at its root. So this part may well not have been done. However .. the next part where it compiles into Docker did work.

So .. if that copy of the env sample is necessary for it to work .. then I'd appreciate some help on how to do that.

Environment Information

I am running the latest version of Docker, and Windows 11 Pro The hardware is an Intel Core 19-10900x 128 gigs of Ram 2 x RTX A6000 GPU's with 48 gigs of Vram each. (I do 3D rendering work).

Screenshots

No response

Relevant log output

No response

Confirmations

[X] I'm running the latest version of the main branch.
[X] I checked existing issues to see if this has already been described.

gaby commented 2 months ago

@SparkieShock That article is ancient from back when you had to build your own Serge to run it.

Once you have Docker Desktop and WSL2 just run this:

docker run -d \
    --name serge \
    -v weights:/usr/src/app/weights \
    -v datadb:/data/db/ \
    -p 8008:8008 \
    ghcr.io/serge-chat/serge:main

SparkieShock commented 2 months ago

Yes I had realized it was an older tutorial .. unfortunately I could not find anything that was more current. I long for the day when one can just hit a 'download' button like a regular app. Github stuff terrifies me cuz I have no idea what potential damage this is doing to my computer. And the readme stuff is all written for those that know coding.

Anyways .. that aside I appreciate the response and thank you for this new docker code. I'll try it out tonight.

Again, many thanks.

gaby commented 1 month ago

@SparkieShock Did you solved your issue?

SparkieShock commented 1 month ago

Hi there. Yes indeed it did. many thanks for your help.

On Sat, Sep 14, 2024 at 5:00 AM Juan Calderon-Perez < @.***> wrote:

@SparkieShock https://github.com/SparkieShock Did you solved your issue?

— Reply to this email directly, view it on GitHub https://github.com/serge-chat/serge/issues/1450#issuecomment-2350965784, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHMKDFTIXMNB7ONZ7WKVVATZWQQMFAVCNFSM6AAAAABN3JSHSGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNJQHE3DKNZYGQ . You are receiving this because you were mentioned.Message ID: @.***>

serge-chat / serge