search

toverainc / willow-inference-server

Open source, local, and self-hosted highly optimized language inference server supporting ASR/STT, TTS, and LLM across WebRTC, REST, and WS
Apache License 2.0
375 stars 33 forks source link

{ "response": "Chatbot not installed or supported" } #148

Open TRONBFFL opened 7 months ago

TRONBFFL commented 7 months ago

I'm not sure what I did wrong in the setup. I dont see anywhere where it tried to download and cache model though. I changed the settings.py support chatbot true - am I missing something?

wis-1 | [2024-02-14 16:11:54 +0000] [94] [DEBUG] FASTAPI: Got chatbot request with text: hello there wis-1 | [2024-02-14 16:11:54 +0000] [94] [WARNING] CHATBOT: Not installed or supported wis-1 | [2024-02-14 16:11:54 +0000] [94] [DEBUG] FASTAPI: Got chatbot response with text: Chatbot not installed or supported

nikito commented 7 months ago

If you are running latest WIS server, chatbot was removed, so these are likely pointing to that fact. 🙂

nikito commented 7 months ago

Adding here, i realized i was thinking of the split_arch branch, it appears main still has support for chatbot (but this will go away soon in favor of using some of the more refined engines out there). You are potentially seeing this error because the card architecture/capability is too low or vram is too low.