[Feature Support] XTuner 已支持百川 2 QLora 微调，单卡可训

baichuan-inc / Baichuan2

A series of large language models developed by Baichuan Intelligent Technology

https://huggingface.co/baichuan-inc

Apache License 2.0

4.09k stars 295 forks source link

[Feature Support] XTuner 已支持百川 2 QLora 微调，单卡可训 #24

Open MJIMMYA opened 1 year ago

MJIMMYA commented 1 year ago

XTuner 已支持百川2 QLora 单卡微调，欢迎加入 Wechat 群交流

git clone https://github.com/internLM/xtuner
cd xtuner
pip install -e .

xtuner train configs/baichuan/baichuan2_7b_base/baichuan2_7b_base_qlora_alpaca_e3.py
# 等同于
# python xtuner/tools/train.py  configs/baichuan/baichuan2_7b_base/baichuan2_7b_base_qlora_alpaca_e3.py

支持多种alpaca、arxiv_gentitle、codealpaca 等数据集开箱即训，会自动下载数据集~

https://github.com/internLM/xtuner

tangsipeng commented 1 year ago

咨询一个问题，我现在的虚拟显存只有8000MiB 请问能否支持百川2 QLora 单卡微调。或者还有没有什么办法可以在节省一些显存