Closed rebcabin closed 6 months ago
The issue is that add_tensor_info
does not support tensor_dtype
Int8DType
. I don't know how this could ever run.
Yes, you need https://github.com/ggerganov/llama.cpp/pull/6045 for this to work.
at the top level outside of mlc (to prevent nested .git repos), I did
conda activate tf
git clone https://github.com/certik/llama.cpp.git
cd llama.cpp
git checkout -t origin/gguf_writer
cd gguf-py
pip install .