search

bofenghuang / vigogne

French instruction-following and chat models
Apache License 2.0
497 stars 47 forks source link

Finetune on 30B LLaMA #10

Closed Awyshw closed 1 year ago

Awyshw commented 1 year ago

When I use torchrun to finetune on A100 2node * 8 GPUs, OOM will appear. Can you release your params?

bofenghuang commented 1 year ago

Hi @Awyshw,

It's in the README. I used 2 RTX8000 of 48Gb

bofenghuang commented 1 year ago

May be https://github.com/tloen/alpaca-lora/pull/361 ?