Closed ZhenYangIACAS closed 10 months ago
How many GPUs do you have? More GPUs will reduce the burden on each individual. Furthermore, the model size and the sequence length are all influential to the conclusion.
8 GPUs, max_words = 512, 7B
That should be enough. You may try the following practices if you meet OOM problem:
--checkpointing
to the main_fiunetune.py
starting command)
Hi, can the code of the current version be full-parameter finetuning on A100 40G ? I have tried, but find the GPU memory only supports 10 layers finetuning. If ok, can you show me the right setting?