Closed wzg-zhuo closed 8 months ago
看错误信息中的这条
error loading model: create_tensor: tensor 'token_embd.weight' has wrong shape; expected 4096, 49953, got 4096, 32000, 1, 1
说明词向量矩阵还是32000(原版词表大小)。你应该检查一下量化前模型的词向量矩阵shape。
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your consideration.
Closing the issue, since no updates observed. Feel free to re-open if you need any further assistance.
提交前必须检查以下项目
问题类型
None
基础模型
None
操作系统
None
详细描述问题
扩充词表后得到merge_hf,替换原来llama2的tokenizer文件,使用llama.cpp量化可以成功,但是加载模型时候报错
依赖情况(代码类问题务必提供)
llama.cpp
运行日志或截图
llm_load_print_meta: freq_base_train = 10000.0 llm_load_print_meta: freq_scale_train = 1 llm_load_print_meta: model type = 7B llm_load_print_meta: model ftype = mostly Q5_0 llm_load_print_meta: model params = 6.74 B llm_load_print_meta: model size = 4.33 GiB (5.52 BPW) llm_load_print_meta: general.name = LLaMA v2 llm_load_print_meta: BOS token = 1 ''
llm_load_print_meta: PAD token = 0 ''
llm_load_print_meta: LF token = 13 '<0x0A>'
llm_load_tensors: ggml ctx size = 0.09 MB
error loading model: create_tensor: tensor 'token_embd.weight' has wrong shape; expected 4096, 49953, got 4096, 32000, 1, 1
llama_load_model_from_file: failed to load model
Traceback (most recent call last):
File "/home/aiproject/aiqa.py", line 2, in
from llm_file import llm,answer_func
File "/home/aiproject/llm_file.py", line 24, in
llm=llm_init()
File "/home/aiproject/llm_file.py", line 18, in llm_init
llm = Llama(model_path=model_path, n_ctx=n_ctx, n_batch=n_batch, n_threads=N_THREADS)
File "/home/miniconda3/lib/python3.10/site-packages/llama_cpp/llama.py", line 365, in init
assert self.model is not None
AssertionError
' llm_load_print_meta: EOS token = 2 '' llm_load_print_meta: UNK token = 0 '