search

tensorflow / models

Models and examples built with TensorFlow
Other
77.02k stars 45.78k forks source link

Execute inference throwing error: Google.Protobuf.Message.DecodeError: Error Parsing Message #6622

Open paviddavid opened 5 years ago

paviddavid commented 5 years ago

Hello all, 

What is the top-level directory of the model you are using /tensorflow/models/research/object_detection
Have I written custom code: No
OS Platform and Distribution: Linux 16.04 LTS
TensorFlow installed from Bazel
TensorFlow version 1.12.0-rc2
Bazel version N/A
CUDA/cuDNN version CUDA release 10.0, V10.0.130, cudNN 7.3.1
GPU model and memory GeForce GTX 980 Ti, 6Gb
Exact command to reproduce: Exact command see above

I am currently using the Tensorflow Object Detection API to train a new model and test it with the KITTI data (as they are provided in within the API).

What have I done? First, I downloaded the pre-trained ResNet101 model (trained on Kitti data) from the model zoo and I ran the inference and everything worked as expected.

However, running the inference on a fine-tuned model, I obtain the following error: 

  File "infer_detections.py", line 96, in <module>
    tf.app.run()
  File "/home/user/.local/lib/python3.5/site-packages/tensorflow/python/platform/app.py", line 125, in run
    _sys.exit(main(argv))
  File "infer_detections.py", line 74, in main
    image_tensor, FLAGS.inference_graph)
  File "/home/user/tensorflow/models/research/object_detection/inference/detection_inference.py", line 71, in build_inference_graph
    graph_def.MergeFromString(graph_content)
google.protobuf.message.DecodeError: Error parsing message

I use this command to do the inference: 

python3.5 infer_detections.py --input_tfrecord_paths=/home/user/tensorflow/models/research/object_detection/data/kitti.record_val.tfrecord --output_tfrecord_path=/home/user/tensorflow/models/research/object_detection/OUTPUT/test/kitti_test.tfrecord --inference_graph=/home/user/tensorflow/models/research/object_detection/OUTPUT/model_12April/export/Servo/1555067137/saved_model.pb --discard_image_pixels

Does anybody know why this error appear? Could it be the case that the error is located within the structure of how the model is saved? The downloaded pre-trained Resnet101 (trained on Kitti data) is organized in the following data structure

|-- faster_rcnn_resnet101_kitti_2018_01_28
    |-- saved_model
        |-- variables 
        |-- saved_model.pb
    |-- checkpoint
    |-- frozen_inference_graph.pb
    |-- model.ckpt.data-00000-of-00001
    |-- model.ckpt.index
    |-- model.ckpt.meta
    |--pipeline.config

However, the data folder that is created in my Output Folder of the training is organized in as follows: 

|-- OutputFolder
    |-- eval_0
        |-- Eventfile
    |-- export
        |-- Servo 
            |-- 1555067137
                 |-- variables
                      |-- variables.data-00000-of-00001
                      |-- variables.index
                 |-- saved_model.pb
    |-- checkpoint 
    |--event File
    |-- graph.pbtxt
    |-- model.ckpt-1961.data-00000-of-00001
    |-- model.ckpt-1961.index
    |-- model.ckpt-1961.meta

Am I using the wrong pb file ? I do not see a frozen_inference_graph.pb file (or similar). Does anybody have a hint how I could solve this?

Thanks a lot in advance and kind regards

tensorflowbutler commented 5 years ago

Thank you for your post. We noticed you have not filled out the following field in the issue template. Could you update them if they are relevant in your case, or leave them as N/A? Thanks. What is the top-level directory of the model you are using Have I written custom code OS Platform and Distribution TensorFlow installed from TensorFlow version Bazel version CUDA/cuDNN version GPU model and memory Exact command to reproduce

paviddavid commented 5 years ago

@tensorflowbutler Updated the post by adding the missing information.

maxpaynestory commented 5 years ago

I am getting the same error with the same folder structure of trained model, but method is different.

graph_def.ParseFromString(graph_content)
annie-surla commented 4 years ago

I have got a similar error, try using the frozen_inference_graph.pb instead of the saved_model.pb. This worked for me. The things is though they are similar files, the internal format is different.

yanglu758 commented 4 years ago

I faced the same error as @paviddavid in the method MergeFromString.

@annie-surla I'm unable to use export_inference_graph.py to generate frozen_inference_graph.pb. The author @pkulzc in issue #8844 suggests that:

placeholder and frozengraph are all TF1 features. In TF2 you should use SavedModel. See exporter_main_v2.py

So I'm under impression that SavedModel consists of the feature of frozen inference graph in TF2.

I'm trying to use infer_detections.py to generate a detection TF record. I have to supply a frozen inference graph (or a Saved Model in TF2) as a parameter to infer_detections.py and I got the same error when I supplied saved_model.pb to infer_detections.py as the argument for --inference_graph . Does anyone know a better approach to obtain a detection TF record perhaps?

Geoyi commented 3 years ago

@yang07ly I am seeing the exact same error, do you find a way to solve it?

yanglu758 commented 3 years ago

No I have not. It seems to be a TF2 feature still in progress for the authors. I edited the source code to print the data I wanted though.

aahlexxx commented 2 years ago

Hello, I am encountering the same error right now, have you guys found a solution about this?