search

ksw0306 / FloWaveNet

A Pytorch implementation of "FloWaveNet: A Generative Flow for Raw Audio"
MIT License
490 stars 109 forks source link

CUDA State error #11

Closed shartoo closed 5 years ago

shartoo commented 5 years ago 
Global Step : 1300, [1, 1300] [Log pdf, Log p(z), Log Det] : [-3.2562 -1.4938  4.75  ]
Global Step : 1400, [1, 1400] [Log pdf, Log p(z), Log Det] : [-3.154  -1.4319  4.5859]
Global Step : 1500, [1, 1500] [Log pdf, Log p(z), Log Det] : [-3.3218 -1.4467  4.7684]
Global Step : 1600, [1, 1600] [Log pdf, Log p(z), Log Det] : [-3.3319 -1.4162  4.7481]
Global Step : 1700, [1, 1700] [Log pdf, Log p(z), Log Det] : [-3.532  -1.4159  4.9479]
Global Step : 1800, [1, 1800] [Log pdf, Log p(z), Log Det] : [-3.0115 -1.549   4.5605]
Global Step : 1900, [1, 1900] [Log pdf, Log p(z), Log Det] : [-3.572  -1.4092  4.9812]
Global Step : 2000, [1, 2000] [Log pdf, Log p(z), Log Det] : [-3.595  -1.4123  5.0073]
Global Step : 2100, [1, 2100] [Log pdf, Log p(z), Log Det] : [-3.3661 -1.4437  4.8097]
Global Step : 2200, [1, 2200] [Log pdf, Log p(z), Log Det] : [-3.43  -1.416  4.846]
1 Epoch Training Loss : -2.8819
Global Step : 2250, [1, 100] [Log pdf, Log p(z), Log Det] : [-3.1498 -1.4728  4.6226]
Global Step : 2250, [1, 200] [Log pdf, Log p(z), Log Det] : [-3.1846 -1.4745  4.6591]
Evaluation Loss : -3.1582
Epoch 1 Model Saved! Loss : -3.1582
Traceback (most recent call last):
  File "H:/workspace/FloWaveNet/train.py", line 244, in <module>
    synthesize(model)
  File "H:/workspace/FloWaveNet/train.py", line 170, in synthesize
    y_gen = model.reverse(z, c).squeeze()
  File "H:\workspace\FloWaveNet\model.py", line 204, in reverse
    c = self.upsample(c)
  File "H:\workspace\FloWaveNet\model.py", line 230, in upsample
    c = f(c)
  File "C:\Python35\lib\site-packages\torch\nn\modules\module.py", line 477, in __call__
    result = self.forward(*input, **kwargs)
  File "C:\Python35\lib\site-packages\torch\nn\modules\conv.py", line 691, in forward
    output_padding, self.groups, self.dilation)
RuntimeError: CuDNN error: CUDNN_STATUS_INTERNAL_ERROR
L0SG commented 5 years ago

I've occasionally experienced this issue during Windows compatibility task a while back (the screen flashes with green for a second then the error). We think that this is a Windows-specific bug of the current CuDNN since we've never encountered this bug in Linux systems. So not much we can do I'm afraid.

shartoo commented 5 years ago

I searched for answer and found it was pytorch bug not this project,thank you .I'll try on ubuntu then