[Question] 明明显存够，却显示out of memory

[luyuhua]

Required prerequisites

• [X] I have read the documentation https://github.com/baichuan-inc/baichuan-7B/blob/HEAD/README.md.
• [X] I have searched the Issue Tracker and Discussions that this hasn't already been reported. (+1 or comment there if it has.)
• [ ] Consider asking first in a Discussion.

Questions

明明显存还有很多没有分配，却显示out of mem OutOfMemoryError: CUDA out of memory. Tried to allocate 86.00 MiB (GPU 0; 24.00 GiB total capacity; 9.07 GiB already allocated; 13.70 GiB free; 9.08 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

Checklist

• [X] I have provided all relevant and necessary information above.
• [X] I have chosen a suitable title for this issue.

[Xls1994]

可以看一下是否是程序内部限制了显存比例呢

[dengxiaotian123]

model = AutoModelForCausalLM.from_pretrained("../baichuan-inc/baichuan-7B", device_map="auto", torch_dtype=torch.float16, trust_remote_code=True)

这样有效

[jaweii]

+1 跑train时内存溢出了