Torch code that I ran with no problem on Google Colab (default configuration) a year ago is now failing with an out-of-memory error. Specifically, calling torch.nn.functional.cosine_similarity() seems to require around double the amount of memory it used to, to the point where this fails:
as the operation requires more than the amount of RAM available (12.7 GB). If I run the same but on a GPU (15 GB of RAM), I get the following error when it crashes:
OutOfMemoryError: CUDA out of memory. Tried to allocate 5.96 GiB (GPU 0; 14.75 GiB total capacity; 12.05 GiB already allocated; 2.57 GiB free; 12.06 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
These errors occur with torch==2.0.1+cu118 and torchvision==0.15.2+cu118. If we downgrade to torch==1.11.0+cu102 and torchvision==0.12.0+cu102, the code runs fine.
Torch code that I ran with no problem on Google Colab (default configuration) a year ago is now failing with an out-of-memory error. Specifically, calling
torch.nn.functional.cosine_similarity()
seems to require around double the amount of memory it used to, to the point where this fails:as the operation requires more than the amount of RAM available (12.7 GB). If I run the same but on a GPU (15 GB of RAM), I get the following error when it crashes:
These errors occur with
torch==2.0.1+cu118
andtorchvision==0.15.2+cu118
. If we downgrade totorch==1.11.0+cu102
andtorchvision==0.12.0+cu102
, the code runs fine.For more see the Issue report on
pytorch
github page https://github.com/pytorch/pytorch/issues/104564