search

lyongo / NWPU-MOC

This is a repository about NWPU-MOC dataset and code.
19 stars 2 forks source link

CUDA memory issue #7

Open mokby opened 2 months ago

mokby commented 2 months ago

When training for about 10 epoch ,error occured, batch size is 2

CUDA out of memory. Tried to allocate 11.47 GiB (GPU 0; 11.99 GiB total capacity; 11.80 GiB already allocated; 0 bytes free; 23.34 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

When test, same error occured, batch size is 1

CUDA out of memory. Tried to allocate 11.47 GiB (GPU 0; 11.99 GiB total capacity; 11.80 GiB already allocated; 0 bytes free; 23.34 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
mokby commented 1 month ago

Solved, dataset problem.