search

martinkersner / train-CRF-RNN

Train CRF-RNN for Semantic Image Segmentation
Other
199 stars 92 forks source link

Unknown enumeration value of "MULTI_STAGE_MEANFIELD" for field "type". #14

Closed snakehaihai closed 8 years ago

snakehaihai commented 8 years ago

WARNING: Logging before InitGoogleLogging() is written to STDERR I0428 23:24:05.063494 19632 solver.cpp:54] Initializing solver from parameters: test_iter: 261 test_interval: 1333 base_lr: 1e-13 display: 50 max_iter: 100000 lr_policy: "fixed" momentum: 0.99 weight_decay: 0.0005 snapshot: 1000 snapshot_prefix: "/home/snake/caffe_crfrnn/train-CRF-RNN-master/snapshot/train" solver_mode: GPU net: "/home/snake/caffe_crfrnn/train-CRF-RNN-master/TVG_CRFRNN_COCO_VOC_TRAIN_3_CLASSES.prototxt" test_initialization: false I0428 23:24:05.063585 19632 solver.cpp:96] Creating training net from net file: /home/snake/caffe_crfrnn/train-CRF-RNN-master/TVG_CRFRNN_COCO_VOC_TRAIN_3_CLASSES.prototxt [libprotobuf ERROR google/protobuf/text_format.cc:245] Error parsing text-format caffe.NetParameter: 213:3: Unknown enumeration value of "MULTI_STAGE_MEANFIELD" for field "type". F0428 23:24:05.064235 19632 upgrade_proto.cpp:932] Check failed: ReadProtoFromTextFile(param_file, param) Failed to parse NetParameter file: /home/snake/caffe_crfrnn/train-CRF-RNN-master/TVG_CRFRNN_COCO_VOC_TRAIN_3_CLASSES.prototxt

Anyidea how to solve this problem?

martinkersner commented 8 years ago

I guess you don't use modified Caffe for CRF-RNN because it couldn't find definition of _MULTI_STAGEMEANFIELD. If you want to train model you need to install Caffe from this repository https://github.com/torrvision/crfasrnn#installation-guide.

snakehaihai commented 8 years ago

Found the problem. Change layers to layer. CONVLUTION to "Convotluion" Its caffe new and old way of prototext problem

snakehaihai commented 8 years ago

Im encountering another problem says that my video ram is not enough. But i reduce the image to 300X200 and 10 label only and batchsize 1. The video ram it appear is 0.5 GB. and my PC has 2GB video ram. May I ask why im receive the error?

0429 19:27:37.009716 3199 net.cpp:298] Network initialization done. I0429 19:27:37.009724 3199 net.cpp:299] Memory required for data: 528391700 I0429 19:27:37.009837 3199 solver.cpp:65] Solver scaffolding done. I0429 19:27:37.009905 3199 caffe.cpp:128] Finetuning from /home/snake/caffe_crfrnn/crfasrnn-master/python-scripts/TVG_CRFRNN_COCO_VOC.caffemodel [libprotobuf WARNING google/protobuf/io/coded_stream.cc:505] Reading dangerously large protocol message. If the message turns out to be larger than 2147483647 bytes, parsing will be halted for security reasons. To increase the limit (or to disable these warnings), see CodedInputStream::SetTotalBytesLimit() in google/protobuf/io/coded_stream.h. [libprotobuf WARNING google/protobuf/io/coded_stream.cc:78] The total number of bytes read was 537968303 I0429 19:27:37.648407 3199 upgrade_proto.cpp:620] Attempting to upgrade input file specified using deprecated V1LayerParameter: /home/snake/caffe_crfrnn/crfasrnn-master/python-scripts/TVG_CRFRNN_COCO_VOC.caffemodel I0429 19:27:37.919209 3199 upgrade_proto.cpp:628] Successfully upgraded file specified using deprecated V1LayerParameter [libprotobuf WARNING google/protobuf/io/coded_stream.cc:505] Reading dangerously large protocol message. If the message turns out to be larger than 2147483647 bytes, parsing will be halted for security reasons. To increase the limit (or to disable these warnings), see CodedInputStream::SetTotalBytesLimit() in google/protobuf/io/coded_stream.h. [libprotobuf WARNING google/protobuf/io/coded_stream.cc:78] The total number of bytes read was 537968303 I0429 19:27:38.654881 3199 upgrade_proto.cpp:620] Attempting to upgrade input file specified using deprecated V1LayerParameter: /home/snake/caffe_crfrnn/crfasrnn-master/python-scripts/TVG_CRFRNN_COCO_VOC.caffemodel I0429 19:27:38.930379 3199 upgrade_proto.cpp:628] Successfully upgraded file specified using deprecated V1LayerParameter I0429 19:27:39.024951 3199 caffe.cpp:211] Starting Optimization I0429 19:27:39.025022 3199 solver.cpp:293] Solving TVG_CRF_RNN_COCO_VOC_TRAIN_3_CLASSES I0429 19:27:39.025032 3199 solver.cpp:294] Learning Rate Policy: fixed F0429 19:27:39.100859 3199 syncedmem.cpp:58] Check failed: error == cudaSuccess (2 vs. 0) out of memory * Check failure stack trace: * @ 0x7f0134bd7daa (unknown) @ 0x7f0134bd7ce4 (unknown) @ 0x7f0134bd76e6 (unknown) @ 0x7f0134bda687 (unknown) @ 0x7f01352ef9e1 caffe::SyncedMemory::to_gpu() @ 0x7f01352eed69 caffe::SyncedMemory::mutable_gpu_data() @ 0x7f01351e2472 caffe::Blob<>::mutable_gpu_data() @ 0x7f013523a070 caffe::BaseConvolutionLayer<>::forward_gpu_gemm() @ 0x7f01352f9f21 caffe::ConvolutionLayer<>::Forward_gpu() @ 0x7f01351f4751 caffe::Net<>::ForwardFromTo() @ 0x7f01351f4ac7 caffe::Net<>::ForwardPrefilled() @ 0x7f013521b279 caffe::Solver<>::Step() @ 0x7f013521bac5 caffe::Solver<>::Solve() @ 0x408f3b train() @ 0x406931 main @ 0x7f01340e9ec5 (unknown) @ 0x40701d (unknown) @ (nil) (unknown) /home/snake/caffe_crfrnn/crfasrnn-master/python-scripts/train.sh: line 1: 3199 Aborted (core dumped) /home/snake/caffe_crfrnn/build/tools/caffe train --solver=/home/snake/caffe_crfrnn/crfasrnn-master/python-scripts/solver.prototxt --weights=/home/snake/caffe_crfrnn/crfasrnn-master/python-scripts/TVG_CRFRNN_COCO_VOC.caffemodel

snakehaihai commented 8 years ago

Found the problem. It shows network 0.5 GB but behind there are things that are not computed inside which is higher than the GTX 680 can compute. After change to a Notebook with 980m, It works.

Engineering-Course commented 8 years ago

@snakehaihai I also have the problem Error parsing text-format caffe.NetParameter: 224:3: Unknown enumeration value of "MULTI_STAGE_MEANFIELD" for field "type".

But I do this like you said, Change layers to layer. CONVLUTION to "Convotluion" It doesn't work. What should I do please?

ghost commented 8 years ago

@Engineering-Course I met same problem, any luck so far please?

Engineering-Course commented 8 years ago

@baoqiangcao It is the problem caused by the different version of Caffe. I changed Caffe with the definition of MULTI_STAGE_MEANFIELD and it works.

ghost commented 8 years ago

@Engineering-Course Thanks, would you share the version works for you? I've been using https://github.com/bittnt/caffe/tree/22babdb4a70eb5829c52fccc417d4b2f916d2748 and got the same error as you described.

snakehaihai commented 8 years ago

For those who face the similar issue. There are 2 ways of caffe prototxt one is capital(start with layers) and another one is with " " (start with layer) for all kind of layers ie convolution, pool, and many others. You need to fix all layers in prototxt in to one of the version. Its difficult to find out but it is quite common. say Layers { type:CONVOLUTION } and Layer { type:"convolution" }

gopi77 commented 7 years ago

The link https://github.com/stoneyang/train-CRF-RNN/commit/ed6a5eaa94b32c09714106b6857ce00d04577a79 has few changes.

When i updated only the file TVG_CRFRNN_COCO_VOC_TRAIN_3_CLASSES.prototxt & ran the command python solve.py 2>&1 | tee train.log, i got this error -> https://github.com/martinkersner/train-CRF-RNN/issues/35

  1. Do i need to update all the files from this commit ? and
  2. what are the steps need to be done again? because this commit modifies 3 classes into 20 again