lmb-freiburg / flownet2

FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks
https://lmb.informatik.uni-freiburg.de/Publications/2017/IMKDB17/
Other
1k stars 318 forks source link

Fails to train model #137

Closed imart302 closed 6 years ago

imart302 commented 6 years ago

Hi. When I was trying to train the model I had some errors.

First I tried with FlowNet2 model. And there are some not defined blobs like "img0_a_org", "img1_a_org", "flow_gt_a_org", "img0_b", "img1_b" and "flow_gt_b". And give me the respective message error that are not defined in the terminal. I used the FlyingChairs Dataset and I prepared with the respective command in the sh file the data folder.

I also tried with FlowNet2-C model. I change the crop width and height for the FlyingChairs dataset in the layers of data augmentation. But at the end, I had this error:

I0519 16:37:41.499851 22163 layer_factory.hpp:77] Creating layer CustomData2 I0519 16:37:41.499871 22163 net.cpp:93] Creating Layer CustomData2 I0519 16:37:41.499881 22163 net.cpp:410] CustomData2 -> blob0 I0519 16:37:41.499894 22163 net.cpp:410] CustomData2 -> blob1 I0519 16:37:41.499905 22163 net.cpp:410] CustomData2 -> blob2 I0519 16:37:41.499917 22163 net.cpp:410] CustomData2 -> unused1 F0519 16:37:41.499956 22163 custom_data_layer.cpp:361] Check failed: mdb_env_open(mdbenv, this->layerparam.data_param().source().c_str(), 0x20000|0x200000, 0664) == 0 (2 vs. 0) mdb_env_open failed Check failure stack trace: @ 0x7f26145a0daa (unknown) @ 0x7f26145a0ce4 (unknown) @ 0x7f26145a06e6 (unknown) @ 0x7f26145a3687 (unknown) @ 0x7f2614d74c9a caffe::CustomDataLayer<>::LayerSetUp() @ 0x7f2614c3a7cc caffe::Net<>::Init() @ 0x7f2614c3b665 caffe::Net<>::Net() @ 0x7f2614dd61b2 caffe::Solver<>::InitTestNets() @ 0x7f2614dd6b0d caffe::Solver<>::Init() @ 0x7f2614dd6dfa caffe::Solver<>::Solver() @ 0x7f2614c0fb83 caffe::Creator_AdamSolver<>() @ 0x40e9ae caffe::SolverRegistry<>::CreateSolver() @ 0x407b42 train() @ 0x4059bc main @ 0x7f261389af45 (unknown) @ 0x4060f1 (unknown) @ (nil) (unknown) Aborted (core dumped)

I do not tried with other datasets because I'm short of space (I'm working remotely). However, in the make-lmdbs.sh file, I cannot locate the files FlyingThings3D_release_TEST.list and FlyingThings3D_release_TRAIN.list.

Is there a solution to this?

imart302 commented 6 years ago

For the second issue Check failed: mdb_env_open(mdbenv, this->layerparam.data_param().source().c_str(), 0x20000|0x200000, 0664) == 0 (2 vs. 0). Seems that the path for the lmdb dataset is incorrect I change the path in the train.prototxt for the global path (/home/user/...../data/) and seems to be correct but now there is another issue: Aborted at 1526835032 (unix time) try "date -d @1526835032" if you are using GNU date PC: @ 0x7f1ccb02ff90 caffe::DecodeData<>() SIGSEGV (@0x7e1cac1e8000) received by PID 364 (TID 0x7f1cbb941700) from PID 18446744072302264320; stack trace: @ 0x7f1cc9b6dcb0 (unknown) @ 0x7f1ccb02ff90 caffe::DecodeData<>() @ 0x7f1ccb0350ac caffe::CustomDataLayerPrefetch<>() @ 0x7f1cc07cc184 start_thread @ 0x7f1cc9c3503d (unknown) @ 0x0 (unknown) Segmentation fault (core dumped)

Any info about this?

nikolausmayer commented 6 years ago

Sorry, the errors don't tell me more than what you already mentioned.

The undefined blobs in the FlowNet2 model are a remnant of our training procedure with two datasets at the same time. If you want to change the model definition, I suggest contacting Eddy Ilg :)

nikolausmayer commented 6 years ago

(closed due to inactivity)