szagoruyko / diracnets

Training Very Deep Neural Networks Without Skip-Connections
https://arxiv.org/abs/1706.00388
586 stars 74 forks source link

cublas runtime error #7

Open eeric opened 7 years ago

eeric commented 7 years ago

On ImageNet set, it occured as following: File "train.py", line 245, in main() File "train.py", line 241, in main engine.train(h, train_loader, opt.epochs, optimizer) File "build/bdist.linux-x86_64/egg/torchnet/engine/engine.py", line 39, in train File "/usr/local/lib/python2.7/dist-packages/torch/optim/sgd.py", line 72, in step loss = closure() File "build/bdist.linux-x86_64/egg/torchnet/engine/engine.py", line 28, in closure File "train.py", line 177, in h y = data_parallel(f, inputs, params, stats, sample[2], np.arange(opt.ngpu)) File "/home/yq/work/face_class/diracnets/diracnet.py", line 51, in data_parallel return f(input, params, stats, mode) File "/home/yq/work/face_class/diracnets/diracnet.py", line 182, in f o = F.linear(o.view(o.size(0), -1), params['fc.weight'], params['fc.bias']) File "/usr/local/lib/python2.7/dist-packages/torch/nn/functional.py", line 449, in linear return state(input, weight) if bias is None else state(input, weight, bias) File "/usr/local/lib/python2.7/dist-packages/torch/nn/functions/linear.py", line 10, in forward output.addmm(0, 1, input, weight.t()) RuntimeError: cublas runtime error : library not initialized at /b/wheel/pytorch-src/torch/lib/THC/THCGeneral.c:394

zhenheny commented 6 years ago

try sudo rm -rf ~/.nv

eeric commented 6 years ago

@zhenheny, thanks, so I didn't pay attention to it at present.