search

yandexdataschool / Practical_DL

DL course co-developed by YSDA, HSE and Skoltech
MIT License
1.54k stars 633 forks source link

week4: Memory problem #43

Open NickVeld opened 4 years ago

NickVeld commented 4 years ago

Google Collab:

CPU

100-th iteration kills runtime with "unknown reason"

CUDA

 0%|          | 110/25000 [00:01<05:47, 71.73it/s]
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-19-adef1f515998> in <module>()
     26 
     27         # use your embedding model to produce feature vector
---> 28         features = embedding(input_tensor) #<YOUR CODE>
     29 
     30         X.append(features)

3 frames

/usr/local/lib/python3.6/dist-packages/torch/nn/functional.py in _max_pool2d(input, kernel_size, stride, padding, dilation, ceil_mode, return_indices)
    485         stride = torch.jit.annotate(List[int], [])
    486     return torch.max_pool2d(
--> 487         input, kernel_size, stride, padding, dilation, ceil_mode)
    488 
    489 max_pool2d = boolean_dispatch(

RuntimeError: CUDA out of memory. Tried to allocate 28.00 MiB (GPU 0; 11.17 GiB total capacity; 10.70 GiB already allocated; 5.81 MiB free; 141.58 MiB cached)
NickVeld commented 4 years ago

CUDA case was fixed by transforming Tensor to ndarray, may be it helps for CPU case