search

sanghoon / pva-faster-rcnn

Demo code for PVANet
https://arxiv.org/abs/1611.08588
Other
650 stars 240 forks source link

Error happens when i train on my own dataset. #47

Open panyiming opened 7 years ago

panyiming commented 7 years ago

when I train with the pretrained model,it comes like this:

I0103 20:21:01.434712 28289 layer_factory.hpp:77] Creating layer proposal AttributeError: 'module' object has no attribute 'ProposalLayer2' Traceback (most recent call last): File "./tools/train_net.py", line 112, in max_iters=args.max_iters) File "/home/panyiming/pva-faster-rcnn/tools/../lib/fast_rcnn/train.py", line 157, in train_net pretrained_model=pretrained_model) File "/home/panyiming/pva-faster-rcnn/tools/../lib/fast_rcnn/train.py", line 43, in init self.solver = caffe.SGDSolver(solver_prototxt) SystemError: NULL result without error in PyObject_Call

But when i train without pretrained model,it can begin to train,but the loss is :+1:

I0103 20:31:44.717337 28915 solver.cpp:238] Iteration 0, loss = nan I0103 20:31:44.717373 28915 solver.cpp:254] Train net output #0: loss_bbox = nan ( 1 = nan loss) I0103 20:31:44.717381 28915 solver.cpp:254] Train net output #1: loss_cls = 87.3365 ( 1 = 87.3365 loss) I0103 20:31:44.717384 28915 solver.cpp:254] Train net output #2: rpn_loss_bbox = nan ( 1 = nan loss) I0103 20:31:44.717391 28915 solver.cpp:254] Train net output #3: rpn_loss_cls = 87.3365 ( 1 = 87.3365 loss) I0103 20:31:44.717409 28915 sgd_solver.cpp:83] Plateau Status: Iteration 0, current minimum_loss = 3.40282e+38 I0103 20:31:44.717417 28915 sgd_solver.cpp:138] Iteration 0, lr = 0.001 I0103 20:31:51.010699 28915 solver.cpp:238] Iteration 20, loss = nan I0103 20:31:51.010735 28915 solver.cpp:254] Train net output #0: loss_bbox = nan ( 1 = nan loss) I0103 20:31:51.010743 28915 solver.cpp:254] Train net output #1: loss_cls = 87.3365 ( 1 = 87.3365 loss) I0103 20:31:51.010749 28915 solver.cpp:254] Train net output #2: rpn_loss_bbox = nan ( 1 = nan loss) I0103 20:31:51.010754 28915 solver.cpp:254] Train net output #3: rpn_loss_cls = 87.3365 ( 1 = 87.3365 loss) I0103 20:31:51.010761 28915 sgd_solver.cpp:138] Iteration 20, lr = 0.001 I0103 20:31:57.685253 28915 solver.cpp:238] Iteration 40, loss = nan I0103 20:31:57.685288 28915 solver.cpp:254] Train net output #0: loss_bbox = nan ( 1 = nan loss) I0103 20:31:57.685295 28915 solver.cpp:254] Train net output #1: loss_cls = 87.3365 ( 1 = 87.3365 loss) I0103 20:31:57.685300 28915 solver.cpp:254] Train net output #2: rpn_loss_bbox = nan ( 1 = nan loss) I0103 20:31:57.685304 28915 solver.cpp:254] Train net output #3: rpn_loss_cls = 87.3365 ( 1 = 87.3365 loss) I0103 20:31:57.685310 28915 sgd_solver.cpp:138] Iteration 40, lr = 0.001 I0103 20:32:04.648226 28915 solver.cpp:238] Iteration 60, loss = nan I0103 20:32:04.648262 28915 solver.cpp:254] Train net output #0: loss_bbox = nan ( 1 = nan loss) I0103 20:32:04.648270 28915 solver.cpp:254] Train net output #1: loss_cls = 87.3365 ( 1 = 87.3365 loss) I0103 20:32:04.648275 28915 solver.cpp:254] Train net output #2: rpn_loss_bbox = nan ( 1 = nan loss) I0103 20:32:04.648280 28915 solver.cpp:254] Train net output #3: rpn_loss_cls = 87.3365 ( 1 = 87.3365 loss) I0103 20:32:04.648288 28915 sgd_solver.cpp:138] Iteration 60, lr = 0.001 I0103 20:32:11.306999 28915 solver.cpp:238] Iteration 80, loss = nan I0103 20:32:11.307044 28915 solver.cpp:254] Train net output #0: loss_bbox = nan ( 1 = nan loss) I0103 20:32:11.307052 28915 solver.cpp:254] Train net output #1: loss_cls = 87.3365 ( 1 = 87.3365 loss) I0103 20:32:11.307057 28915 solver.cpp:254] Train net output #2: rpn_loss_bbox = nan ( 1 = nan loss) I0103 20:32:11.307065 28915 solver.cpp:254] Train net output #3: rpn_loss_cls = 87.3365 ( 1 = 87.3365 loss) I0103 20:32:11.307072 28915 sgd_solver.cpp:138] Iteration 80, lr = 0.001

solemnrole commented 7 years ago

AttributeError: 'module' object has no attribute 'ProposalLayer2' ; first,check your caffe.proto,proposal_layer.cu,this two files,make sure they are correct,and declared. then,compile caffe

MyVanitar commented 7 years ago

@PANYIMING @solemnrole

To start training on my own dataset, I have these questions:

yzhang123 commented 7 years ago

@VanitarNordic Coco has 80 labels and contains all labels of Voc. you can train using python tools/train_net.py --gpu 0 --solver models/pvanet/example_train/solver.prototxt  --weight models/pvanet/pretrained/pva9.1_pretrained_no_fc6.caffemodel --iters 100000 --cfg models/pvanet/cfgs/train.yml --imdb coco_2014_train+voc_2012_trainval