search

marella / ctransformers

Python bindings for the Transformer models implemented in C/C++ using GGML library.
MIT License
1.76k stars 137 forks source link

Unsupported Model : Zephyr 'stablelm' GGUF #196

Open Jonathanjordan21 opened 6 months ago

Jonathanjordan21 commented 6 months ago

I wonder why Zephyr is not yet supported in ctransformers. 

from ctransformers import AutoModelForCausalLM

m = AutoModelForCausalLM.from_pretrained('TheBloke/stablelm-zephyr-3b-GGUF', model_file='stablelm-zephyr-3b.Q4_K_M.gguf')

The above code actually downloaded the file from huggingface repo, but it output this error

RuntimeError: Failed to create LLM 'stablelm' from '/root/.cache/huggingface/hub/models--TheBloke--stablelm-zephyr-3b-GGUF/blobs/74b2613b6e89d904a2ea38d56d233e4d2ca2fe663844b2e7aa90e769d359061b'.