search

chuanqi305 / SqueezeNet-SSD

SSD detection network of SqueezeNet, with pretrained weights on VOC0712 and mAP=0.643
64 stars 35 forks source link

Errors when trying to retrain the model on custom data set #2

Closed arisha07 closed 6 years ago

arisha07 commented 7 years ago

Hello,

I am trying to retrain this model on my own custom dataset using intel caffe but i am running into errors.

  1. when I train on GPU: F1010 05:26:52.085708 27020 mkl_split_layer.cpp:234] Not Implemented Yet Check failure stack trace: @ 0x7f36a7c455cd google::LogMessage::Fail() @ 0x7f36a7c47433 google::LogMessage::SendToLog() @ 0x7f36a7c4515b google::LogMessage::Flush() @ 0x7f36a7c47e1e google::LogMessageFatal::~LogMessageFatal() @ 0x7f36a83bfee0 caffe::MKLSplitLayer<>::Forward_gpu() @ 0x7f36a85eba62 caffe::Net<>::ForwardFromTo() @ 0x7f36a85ebb77 caffe::Net<>::Forward() @ 0x7f36a8592f6a caffe::Solver<>::Solve() @ 0x558276ee9646 train() @ 0x558276ee0f4f main @ 0x7f36a62d6830 __libc_start_main @ 0x558276ee1bc9 _start @ (nil) (unknown)

  2. When I train on CPU: F1010 04:56:10.058949 12450 mkl_concat_layer.cpp:176] Check failed: e == E_SUCCESS (-1 vs. 0) Check failure stack trace: @ 0x7fac75fa15cd google::LogMessage::Fail() @ 0x7fac75fa3433 google::LogMessage::SendToLog() @ 0x7fac75fa115b google::LogMessage::Flush() @ 0x7fac75fa3e1e google::LogMessageFatal::~LogMessageFatal() @ 0x7fac764d2c54 caffe::MKLConcatLayer<>::Forward_cpu() @ 0x7fac766c61b2 caffe::Net<>::ForwardFromTo() @ 0x7fac766c63c7 caffe::Net<>::Forward() @ 0x7fac7667235a caffe::Solver<>::Solve() @ 0x564ae0d7ce63 train() @ 0x564ae0d74b8f main @ 0x7fac74632830 __libc_start_main @ 0x564ae0d75809 _start @ (nil) (unknown)

Please let me know how did you train this model. I must be doing something wrong. Any help is greatly appreciated! Thank you

chuanqi305 commented 6 years ago

@arisha07 Seems that there are some errors in your caffe. Recompile caffe or test on another computer.

arisha07 commented 6 years ago

Hello @chuanqi305,

Yes, I was trying to use intel caffe thats why i was getting those errors. But using weiliu-caffe fixed it. Thank you for the reply.

On Tue, Oct 31, 2017 at 7:31 AM, chuanqi305 notifications@github.com wrote:

@arisha07 https://github.com/arisha07 Seems that there are some errors in your caffe. Recompile caffe or test on another computer.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/chuanqi305/SqueezeNet-SSD/issues/2#issuecomment-340780353, or mute the thread https://github.com/notifications/unsubscribe-auth/AWQDz2TtICJs2U5ETTQ1AN7VZ6nDZjfxks5sxy83gaJpZM4P0d7n .