search

hyperonym / basaran

Basaran is an open-source alternative to the OpenAI text completion API. It provides a compatible streaming API for your Hugging Face Transformers-based text generation models.
MIT License
1.29k stars 81 forks source link

Error when Running Vicuna's FastChat Model without GPU #223

Closed davyeu closed 12 months ago

davyeu commented 1 year ago

I am new to Vicuna.

I wish to use their open source model to train my dataset.

I don't have a GPU in my computer, so I wanted to use their RESTful API Server. I used Windows PowerShell for the commands below.

According to their explanation (https://github.com/lm-sys/FastChat/blob/main/docs/openai_api.md)

First, I launched the command

python3 -m fastchat.serve.controller

. Then, it opened a localhost for me. I opened it in my browser and it displayed the following message:

{"detail":"Not Found"}.

Next, I opened a new PowerShell window and ran their second command:

python3 -m fastchat.serve.model_worker --model-path lmsys/vicuna-7b-v1.3 However, I encountered the following error:

"AssertionError: Torch not compiled with CUDA enabled".

Does this error occur because I do not have a GPU in my computer?

peakji commented 12 months ago

Hi @davyeu, the issue is not related to this project. I would suggest to find help in FastChat.