search

mlabonne / llm-course

Course to get into Large Language Models (LLMs) with roadmaps and Colab notebooks.
https://mlabonne.github.io/blog/
Apache License 2.0
38.9k stars 4.11k forks source link

File not found error while using GGUF in AutoQuant #85

Open Goekdeniz-Guelmez opened 4 months ago

Goekdeniz-Guelmez commented 4 months ago

Hey, i want to quantize my Qwen2 model but it seems then the files are not found even though it clones and installs llama.cpp correctly. When quantizing the mode i get this:

python3: can't open file '/content/llama.cpp/convert-hf-to-gguf.py': [Errno 2] No such file or directory
/bin/bash: line 1: ./llama.cpp/quantize: No such file or directory
/bin/bash: line 1: ./llama.cpp/quantize: No such file or directory

This happens since the new versions of llama.cpp came out, they removed the quantize directory, I think its somewhere gguf-py directory?

GPU: T4

mlabonne commented 4 months ago

Thanks for your feedback, this is fixed.