Hardware requirements for Training/Finetuning?

THUDM / LongWriter

LongWriter: Unleashing 10,000+ Word Generation from Long Context LLMs

Apache License 2.0

1.46k stars 123 forks source link

Hardware requirements for Training/Finetuning? #33

Open CHesketh76 opened 2 weeks ago

CHesketh76 commented 2 weeks ago

I have a 3070Ti and was wondering if running this training pipeline on consumer grade hardware is possible. If not, then what is the recommend hardware requirement and cost of training?

bys0318 commented 1 week ago

Hi, I'm sorry but I don't think RTX 3070Ti has sufficient memory size for training or running LongWriter model. We train on 8xH800 (80GB) for full fine-tuning (LoRA and quantization may reduce the memory utilization).