第一次跑大模型train.py，两张24G显卡，baichuan-7b，出现OutOfMemoryError是正常的吗，求大佬指教

yangjianxin1 / Firefly

Firefly: 大模型训练工具，支持训练Qwen2.5、Qwen2、Yi1.5、Phi-3、Llama3、Gemma、MiniCPM、Yi、Deepseek、Orion、Xverse、Mixtral-8x7B、Zephyr、Mistral、Baichuan2、Llma2、Llama、Qwen、Baichuan、ChatGLM2、InternLM、Ziya2、Vicuna、Bloom等大模型

5.85k stars 525 forks source link

第一次跑大模型train.py，两张24G显卡，baichuan-7b，出现OutOfMemoryError是正常的吗，求大佬指教 #56

Open kunzeng-ch opened 1 year ago

kunzeng-ch commented 1 year ago

torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 3.80 GiB (GPU 1; 23.30 GiB total capacity; 19.62 GiB already allocated; 3.07 GiB free; 19.63 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

LittleXu1998 commented 1 year ago

正常

kunzeng-ch commented 1 year ago

谢谢大佬