search

hyintell / BLOOM-fine-tuning

Finetune BLOOM
40 stars 10 forks source link

微调7B的大概用多少GPU #2

Open tigerbaby1515 opened 1 year ago

roy-shih commented 1 year ago

In my simple test, the size of the "bloomz-7b1" model file is 14.14G(9.98G+4.16G) When I load it to the GPU and use the "tatsu-lab/alpaca" data to fine-tune (size: 24.2 MB) is about 38GB RAM (FP16, A100 40G GPU is OOM)

wakeb98 commented 1 year ago

In my simple test, the size of the "bloomz-7b1" model file is 14.14G(9.98G+4.16G) When I load it to the GPU and use the "tatsu-lab/alpaca" data to fine-tune (size: 24.2 MB) is about 38GB RAM (FP16, A100 40G GPU is OOM)

did you solve the issue?