What is minimum resource requirement to fine-tuning Llama-2-7B-32K-beta model.

togethercomputer / OpenChatKit

Apache License 2.0

9.01k stars 1.01k forks source link

What is minimum resource requirement to fine-tuning Llama-2-7B-32K-beta model. #176

Open zzbin0 opened 8 months ago

zzbin0 commented 8 months ago

Hi, What is minimum resource requirement to fine-tuning Llama-2-7B-32K-beta model with finetune_llama-2-7b-32k-mqa.sh script. Such as disk, memory and vRAM. Look forward to your response. Best regards.