search

marella / chatdocs

Chat with your documents offline using AI.
MIT License
683 stars 97 forks source link

model_file entry in chatdocs.yml should not include .safetensors at end #67

Closed lightnin closed 1 year ago

lightnin commented 1 year ago

According to the README and the default chatdocs.yml, model file entries appear as following: model_file: Wizard-Vicuna-7B-Uncensored-GPTQ-4bit-128g.no-act-order.safetensors

This results in a file not found error. Remove the ".safetensors" and thinks start working better.

References: https://huggingface.co/TheBloke/guanaco-33B-GPTQ/discussions/12#649eec3323c9440af9dd99ff

marella commented 1 year ago

The .safetensors extension is removed internally. The issue was that the model file in TheBloke's repo was renamed from Wizard-Vicuna-7B-Uncensored-GPTQ-4bit-128g.no-act-order.safetensors to Wizard-Vicuna-7B-Uncensored-GPTQ-4bit-128g.no-act.order.safetensors which I updated now and will release in the next version. So adding the following change to your chatdocs.yml file will resolve the issue:

gptq:
  model_file: Wizard-Vicuna-7B-Uncensored-GPTQ-4bit-128g.no-act.order.safetensors