RuntimeError: CUDA out of memory

[DanBigioi]

Hey guys! I get the following error when trying to convert my spectrograms to audio when using melGAN:

RuntimeError: CUDA out of memory. Tried to allocate 2.00 MiB (GPU 0; 4.00 GiB total capacity; 16.61 MiB already allocated; 0 bytes free; 26.00 MiB reserved in total by PyTorch)

This is the code that's causing it

if torch.cuda.is_available():
    vocoder = vocoder.cuda()
    mel = mel.cuda()

with torch.no_grad():
    audio = vocoder.inference(mel)

Its taken straight from the notebook. Any solutions to this? The entire error message looks like this:

RuntimeError                              Traceback (most recent call last)
<ipython-input-9-d382b2c2e31f> in <module>
      4 
      5 with torch.no_grad():
----> 6     audio = vocoder.inference(mel)

~/.cache\torch\hub\seungwonpark_melgan_master\model\generator.py in inference(self, mel)
     70         mel = torch.cat((mel, zero), dim=2)
     71 
---> 72         audio = self.forward(mel)
     73         audio = audio.squeeze() # collapse all dimension except time axis
     74         audio = audio[:-(hop_length*10)]

~/.cache\torch\hub\seungwonpark_melgan_master\model\generator.py in forward(self, mel)
     46     def forward(self, mel):
     47         mel = (mel + 5.0) / 5.0 # roughly normalize spectrogram
---> 48         return self.generator(mel)
     49 
     50     def eval(self, inference=False):

~\AppData\Local\Continuum\anaconda3\envs\Transformer TTS\lib\site-packages\torch\nn\modules\module.py in __call__(self, *input, **kwargs)
    548             result = self._slow_forward(*input, **kwargs)
    549         else:
--> 550             result = self.forward(*input, **kwargs)
    551         for hook in self._forward_hooks.values():
    552             hook_result = hook(self, input, result)

~\AppData\Local\Continuum\anaconda3\envs\Transformer TTS\lib\site-packages\torch\nn\modules\container.py in forward(self, input)
     98     def forward(self, input):
     99         for module in self:
--> 100             input = module(input)
    101         return input
    102 

~\AppData\Local\Continuum\anaconda3\envs\Transformer TTS\lib\site-packages\torch\nn\modules\module.py in __call__(self, *input, **kwargs)
    548             result = self._slow_forward(*input, **kwargs)
    549         else:
--> 550             result = self.forward(*input, **kwargs)
    551         for hook in self._forward_hooks.values():
    552             hook_result = hook(self, input, result)

~\AppData\Local\Continuum\anaconda3\envs\Transformer TTS\lib\site-packages\torch\nn\modules\conv.py in forward(self, input, output_size)
    645         return F.conv_transpose1d(
    646             input, self.weight, self.bias, self.stride, self.padding,
--> 647             output_padding, self.groups, self.dilation)
    648 
    649 

RuntimeError: CUDA out of memory. Tried to allocate 2.00 MiB (GPU 0; 4.00 GiB total capacity; 16.61 MiB already allocated; 0 bytes free; 26.00 MiB reserved in total by PyTorch)

[DanBigioi]

I tried running the autoregressive model with WaveRNN instead and I get the same error at inference stage. I dont think its a GPU issue either as I tried running the notebooks on two separate machines and I obtained the same error.

[aayushkubb]

Hey can you show the nvidia-smi trace when you run the training and also print out Cuda visible devices?

I am able to run the notebook code easily without any changes, if you still have an issue, lower your batch size.

[DanBigioi]

@aayushkubb This is the nvidia-smi trace:

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 451.22       Driver Version: 451.22       CUDA Version: 11.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name            TCC/WDDM | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 105... WDDM  | 00000000:01:00.0 Off |                  N/A |
| N/A   62C    P8    N/A /  N/A |   4031MiB /  4096MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A     13860      C   ...ransformer TTS\python.exe    N/A      |
+-----------------------------------------------------------------------------+

And then this is the error from Cuda

<ipython-input-7-e00f39c1b966> in <module>
     14 
     15 with torch.no_grad():
---> 16     audio = vocoder.inference(mel)

~/.cache\torch\hub\seungwonpark_melgan_master\model\generator.py in inference(self, mel)
     70         mel = torch.cat((mel, zero), dim=2)
     71 
---> 72         audio = self.forward(mel)
     73         audio = audio.squeeze() # collapse all dimension except time axis
     74         audio = audio[:-(hop_length*10)]

~/.cache\torch\hub\seungwonpark_melgan_master\model\generator.py in forward(self, mel)
     46     def forward(self, mel):
     47         mel = (mel + 5.0) / 5.0 # roughly normalize spectrogram
---> 48         return self.generator(mel)
     49 
     50     def eval(self, inference=False):

~\AppData\Local\Continuum\anaconda3\envs\Transformer TTS\lib\site-packages\torch\nn\modules\module.py in __call__(self, *input, **kwargs)
    548             result = self._slow_forward(*input, **kwargs)
    549         else:
--> 550             result = self.forward(*input, **kwargs)
    551         for hook in self._forward_hooks.values():
    552             hook_result = hook(self, input, result)

~\AppData\Local\Continuum\anaconda3\envs\Transformer TTS\lib\site-packages\torch\nn\modules\container.py in forward(self, input)
     98     def forward(self, input):
     99         for module in self:
--> 100             input = module(input)
    101         return input
    102 

~\AppData\Local\Continuum\anaconda3\envs\Transformer TTS\lib\site-packages\torch\nn\modules\module.py in __call__(self, *input, **kwargs)
    548             result = self._slow_forward(*input, **kwargs)
    549         else:
--> 550             result = self.forward(*input, **kwargs)
    551         for hook in self._forward_hooks.values():
    552             hook_result = hook(self, input, result)

~/.cache\torch\hub\seungwonpark_melgan_master\model\res_stack.py in forward(self, x)
     27     def forward(self, x):
     28         for block, shortcut in zip(self.blocks, self.shortcuts):
---> 29             x = shortcut(x) + block(x)
     30         return x
     31 

~\AppData\Local\Continuum\anaconda3\envs\Transformer TTS\lib\site-packages\torch\nn\modules\module.py in __call__(self, *input, **kwargs)
    548             result = self._slow_forward(*input, **kwargs)
    549         else:
--> 550             result = self.forward(*input, **kwargs)
    551         for hook in self._forward_hooks.values():
    552             hook_result = hook(self, input, result)

~\AppData\Local\Continuum\anaconda3\envs\Transformer TTS\lib\site-packages\torch\nn\modules\conv.py in forward(self, input)
    210                             _single(0), self.dilation, self.groups)
    211         return F.conv1d(input, self.weight, self.bias, self.stride,
--> 212                         self.padding, self.dilation, self.groups)

[DanBigioi]

@aayushkubb The batch size is most probable to be the cause. Do you know where I would go to change it?

[cfrancesco]

Hi, one quick thing you can try is switching from GPU to CPU by simply removing those lines. Unless you're predicting on a batch, batch size won't make any difference.

edit: you can always check out the original repos, in case this issue was already brought up