search

xNul / chat-llama-discord-bot

A Discord Bot for chatting with LLaMA, Vicuna, Alpaca, MPT, or any other Large Language Model (LLM) supported by text-generation-webui or llama.cpp.
https://discord.gg/TcRGDV754Y
MIT License
118 stars 23 forks source link

Error loading quantized models #10

Closed dillfrescott closed 1 year ago

dillfrescott commented 1 year ago 
(textgen) C:\Users\micro\Downloads\text-generation-webui>python bot.py --model vicuna-13B-1.1-GPTQ-4bit-128g\vicuna-13B-1.1-GPTQ-4bit-128g.safetensors --wbits 4 --groupsize 128 --model-dir vicuna-13B-1.1-GPTQ-4bit-128g
Loading vicuna-13B-1.1-GPTQ-4bit-128g\vicuna-13B-1.1-GPTQ-4bit-128g.safetensors...
Could not find the quantized model in .pt or .safetensors format, exiting...
dillfrescott commented 1 year ago

I have tried with and without the --model-dir flag as well, to no avail. something is fundamentally wrong

dillfrescott commented 1 year ago

Disregard I missed the part of having to put it in the models folder first