search

Blealtan / RWKV-LM-LoRA

RWKV is a RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.
Apache License 2.0
406 stars 40 forks source link

微调后运行，AttributeError: 'types.SimpleNamespace' object has no attribute 'emb' #14

Open aolerv opened 1 year ago

aolerv commented 1 year ago

Run prompt... Traceback (most recent call last): File "/home/v/下载/RWKV-LM-LoRA-main/RWKV-v4neo/chat.py", line 187, in out = run_rnn(tokenizer.tokenizer.encode(init_prompt)) File "/home/v/下载/RWKV-LM-LoRA-main/RWKV-v4neo/chat.py", line 157, in run_rnn current_state = model.forward(model_tokens, current_state, preprocess_only = True) File "/home/v/下载/RWKV-LM-LoRA-main/RWKV-v4neo/src/model_run.py", line 225, in forward x = w.emb.weight[ctx[-1]] AttributeError: 'types.SimpleNamespace' object has no attribute 'emb' v@v-virtual-machine:~/下载/RWKV-LM-LoRA-main/RWKV-v4neo$

Blealtan commented 1 year ago

正常的模型文件不应该出现这种问题。你是不是直接把LoRA的增量checkpoint路径填进了模型路径 MODEL_NAME，结果只加载了lora而没有加载基座？应该同时加载二者，LoRA的增量checkpoint路径是写在 MODEL_LORA 里的。