Open Beatlesso opened 6 months ago
Seems that you are having a hanging issue. Could you refer to this issue and see if the solution helps?
Seems that you are having a hanging issue. Could you refer to this issue and see if the solution helps?
Thanks, that solved my problem. One more question for you, what is the minimum amount of GPU memory needed to run this pruning algorithm to prune Llama2? 4090 doesn't seem to be enough memory, and multiple GPUs don't solve the problem.
Based on my experience, a minimum of 2 A100 80GB GPUs are required training with the Llama2-7b model with a maximum sequence length of 2048.
After I run pruning.sh, the command line outputs "Starting training..." and the config message. However, I did not get any other output after running it overnight. In fact, I'm running the program on two 4090 GPU, both of which have 16.6GB of memory and consume only 25W, as if the program is not running at all. This makes me think that it's not a slow running problem, but that there may be something else going on. Is this warning "The memory monitor only works on CUDA devices, but the model is on cpu." causing the problem? Can you help me figure out the cause of the problem? Below is my output:
GPU information
running bash