Bug: converting model from HF to GGUF gives error

[thesyntaxinator]

What happened?

When trying to convert my quantized LoRA model from HF to GGUF using convert-hf-to-gguf.py I get the below error: ValueError: Can not map tensor 'model.layers.0.mlp.down_proj.qweight'

Expected behavior: Conversion should have succeeded.

Name and Version

convert-hf-to-gguf.py, python 3.11.6

What operating system are you seeing the problem on?

No response

Relevant log output

python convert-hf-to-gguf.py <HF model directory>
INFO:hf-to-gguf:Loading model: <redacted>
INFO:gguf.gguf_writer:gguf: This GGUF file is for Little Endian only
INFO:hf-to-gguf:Set model parameters
INFO:hf-to-gguf:gguf: context length = 8192
INFO:hf-to-gguf:gguf: embedding length = 4096
INFO:hf-to-gguf:gguf: feed forward length = 14336
INFO:hf-to-gguf:gguf: head count = 32
INFO:hf-to-gguf:gguf: key-value head count = 8
INFO:hf-to-gguf:gguf: rope theta = 500000.0
INFO:hf-to-gguf:gguf: rms norm epsilon = 1e-05
INFO:hf-to-gguf:gguf: file type = 1
INFO:hf-to-gguf:Set model tokenizer
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
INFO:gguf.vocab:Adding 280147 merge(s).
INFO:gguf.vocab:Setting special token type bos to 128000
INFO:gguf.vocab:Setting special token type eos to 128009
INFO:gguf.vocab:Setting special token type pad to 128256
INFO:gguf.vocab:Setting chat_template to {% set loop_messages = messages %}{% for message in loop_messages %}{% set content = '<|start_header_id|>' + message['role'] + '<|end_header_id|>

'+ message['content'] | trim + '<|eot_id|>' %}{% if loop.index0 == 0 %}{% set content = bos_token + content %}{% endif %}{{ content }}{% endfor %}{% if add_generation_prompt %}{{ '<|start_header_id|>assistant<|end_header_id|>

' }}{% endif %}
INFO:hf-to-gguf:Exporting model...
INFO:hf-to-gguf:gguf: loading model weight map from 'model.safetensors.index.json'
INFO:hf-to-gguf:gguf: loading model part 'model-00001-of-00002.safetensors'
INFO:hf-to-gguf:token_embd.weight,           torch.float16 --> F16, shape = {4096, 128257}
INFO:hf-to-gguf:blk.0.attn_norm.weight,      torch.float16 --> F32, shape = {4096}
Traceback (most recent call last):
  File "llama.cpp/convert-hf-to-gguf.py", line 3096, in <module>
    main()
  File "llama.cpp/convert-hf-to-gguf.py", line 3091, in main
    model_instance.write()
  File "llama.cpp/convert-hf-to-gguf.py", line 330, in write
    self.write_tensors()
  File "llama.cpp/convert-hf-to-gguf.py", line 1402, in write_tensors
    super().write_tensors()
  File "llama.cpp/convert-hf-to-gguf.py", line 267, in write_tensors
    for new_name, data in ((n, d.squeeze().numpy()) for n, d in self.modify_tensors(data_torch, name, bid)):
                                                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "llama.cpp/convert-hf-to-gguf.py", line 1399, in modify_tensors
    return [(self.map_tensor_name(name), data_torch)]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "llama.cpp/convert-hf-to-gguf.py", line 185, in map_tensor_name
    raise ValueError(f"Can not map tensor {name!r}")
ValueError: Can not map tensor 'model.layers.0.mlp.down_proj.qweight'

[tristandruyen]

What model are you trying to convert ? Each model family has to get support....

[github-actions[bot]]

This issue was closed because it has been inactive for 14 days since being marked as stale.