search

ggerganov / llama.cpp

LLM inference in C/C++
MIT License
68.55k stars 9.85k forks source link

Bug: ERROR-hf-to-gguf #9483

Closed xyangyan closed 2 months ago

xyangyan commented 2 months ago

What happened?

python convert_hf_to_gguf.py /llama3.1_pruned_0.6_4-29 image We executed the conversion command, but encountered the above error. Here, 'llama3.1_pruned_0.6_4-29' is the version we used after structured pruning. I would like to ask how to operate to convert it to the gguf format, and which file's code needs to be adapted?

Name and Version

version: 3735 (df4b7945) built with cc (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0 for x86_64-linux-gnu

What operating system are you seeing the problem on?

Linux

Relevant log output

No response

Galunid commented 2 months ago

Depending on what you did to the architecture you need to change https://github.com/ggerganov/llama.cpp/blob/822b6322dea704110797a5671fc80ae39ee6ac97/convert_hf_to_gguf.py#L1490-L1635

Adding "LlamaPrunedForCausalLM" to the register in the first line could be enough. If it's not then you need to implement a separate class with your changes that could inherit after LlamaModel