search

BlinkDL / RWKV-LM

RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.
Apache License 2.0
12.48k stars 848 forks source link

lora训练时出错 #196

Closed surviveMiao closed 11 months ago

surviveMiao commented 11 months ago

我运行RWKV-v4neo下的train.py文件，但是它启动不起来，自己断 还没反应 显示的界面 QJ45B5LV2FFD J925`CSR2W 1~AG7EWS_}E{_N0RTEXXCYE 自己的参数设置如图： _R~}$7SI1$)GTY9)(2VPWS 头秃哇。。。

BlinkDL commented 11 months ago

可能是内存不足

surviveMiao commented 11 months ago

对的对的，是内存不足的原因。 没想到微调7b模型 128的内存都不够用。

surviveMiao commented 11 months ago

可能是内存不足

感谢您的回复，谢谢