Can Nivdia 3090 with 24G video memory support finetune?

lxe / simple-llm-finetuner

Simple UI for LLM Model Finetuning

MIT License

2.05k stars 132 forks source link

Can Nivdia 3090 with 24G video memory support finetune? #7

Open pczzy opened 1 year ago

lxe commented 1 year ago

Yes! You can go even lower if you use shorter sequence length.

wizztjh commented 1 year ago

How about 3060?

Shiro836 commented 1 year ago

I was able to use 15 batch size with 200 max length on 4090. 'train_samples_per_second': 3.713. Couldn't get higher batch size for some reason, even though only 17 gb was taken, and 7 was available.

leszekhanusz commented 1 year ago

I hope finetuning will work on my RTX 3080 10GB when deepspeed will be implemented...