search

KaiyangZhou / deep-person-reid

Torchreid: Deep learning person re-identification in PyTorch.
https://kaiyangzhou.github.io/deep-person-reid/
MIT License
4.33k stars 1.15k forks source link

Always getting CUDA out of memory #556

Closed kailashhambarde closed 1 year ago

kailashhambarde commented 1 year ago

Always getting CUDA out of memory Command used: CUDA_VISIBLE_DEVICES=0,1,2,3,4 python main.py even i tried to see if it works with bs=1, but no success self.padding, self.dilation, self.groups) RuntimeError: CUDA out of memory. Tried to allocate 2.00 MiB (GPU 0; 47.54 GiB total capacity; 194.45 MiB already allocated; 4.88 MiB free; 686.00 MiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

kailashhambarde commented 1 year ago

Problem solved: I excluded device 0 and used the following command: CUDA_VISIBLE_DEVICES=1,2,3,4. I'm not exactly sure why it worked, but it did.