Open w32zhong opened 1 year ago
I have checked Model weights loading, the only thing different is that HF model.layers.*.self_attn.rotary_emb.inv_freq
are not loaded: https://github.com/OpenBMB/ModelCenter/blob/828491f12284ba5e199a4db1f370fcd44c70e0f9/model_center/layer/position_embedding.py#L303
But looks like their values should be the same.
I would appreciate anyone can help me out. Thanks!
Describe the bug
I used a verified LLaMA 7B hg checkpoint, and used a single thread bmb to do inference. But the output are just random gibberish. Not sure why?
Minimal steps to reproduce
My checkpoint conversion and inference code is:
Expected behavior
I expect the output to be fluent and meaningful English.
Screenshots
actual output:
Environment:
bmtrain 0.2.2 torch 2.1.0.dev20230630+cu121 nvidia/label/cuda-12.1.1