一张A100-SXM4-40GB 能够用来finetuning phoenix-inst-chat-7b-int4 吗

FreedomIntelligence / LLMZoo

⚡LLM Zoo is a project that provides data, models, and evaluation benchmark for large language models.⚡

Apache License 2.0

2.93k stars 201 forks source link

Open porridgeshoes opened 1 year ago

porridgeshoes commented 1 year ago

我用一张A100-SXM4-40GB尝试finetue时，将batchsize设置成1，但内存还是爆了。难道单卡A100真的跑不起来吗

zhjohnchan commented 1 year ago

Hi @porridgeshoes,

单卡目前还跑不起来，需要用更多的一些技巧，可以参考现有的仓库.

Best, Zhihong