search

s9xie / hed

code for Holistically-Nested Edge Detection
Other
1.79k stars 528 forks source link

I can train smoothly，but when I test, there is a errror: #52

Open tyl569 opened 6 years ago

tyl569 commented 6 years ago

Hello, I can train smoothly，but when I test, there is a errror: I1225 18:53:09.767861 16202 net.cpp:299] Memory required for data: 1097870692 F1225 18:53:10.263083 16202 syncedmem.cpp:19] Check failed: error == cudaSuccess (29 vs. 0) driver shutting down Check failure stack trace: Aborted (core dumped)

Thank you

tyl569 commented 6 years ago

Is this because GPU is not enough memory？

rribani commented 6 years ago

I'm facing a very similar problem, but in my case it happens only with the latest caffe version:

syncedmem.cpp:71] Check failed: error == cudaSuccess (2 vs. 0) out of memory Check failure stack trace: Process finished with exit code 134 (interrupted by signal 6: SIGABRT)

With the old caffe it works. I'm using cuda 9.0.

wangjiaod commented 6 years ago

I meet the same problem,please tell the answaer

aditipanda commented 5 years ago

I am new to Caffe, and am also facing this problem.

I read in another git thread (https://github.com/alexgkendall/caffe-segnet/issues/57#issuecomment-276044793) that calling caffe.reset_all() might help, but have no idea where to place this line of code.

Secondly, this git thread also speaks about MATLAB interface of Caffe. Now in the installation instruction file of HED, it is written "Python and/or MATLAB(optional)". So that means MATLAB is optional. Accordingly, I also trained the network on my GPU which does not have MATLAB. So does this mean by any chance that this training isn't optimal, as the MATLAB interface could not be used?

Also, while testing, since we are using MATLAB to create a network by the command "caffe.Net(model_name, weights)", it means MATLAB interface is compulsory for testing.

But in that case it should also be compulsory for training, as we are using other commands like "caffe.set_gpu(0)". They are also for MATLAB interface. Correct me if I am wrong.

I am thinking I have to re-train the network after installing MATLAB on my Ubuntu workstation.

Was any of you able to solve it? @wangjiaod @rribani @tyl569