microsoft / MMdnn

MMdnn is a set of tools to help users inter-operate among different deep learning frameworks. E.g. model conversion and visualization. Convert models between Caffe, Keras, MXNet, Tensorflow, CNTK, PyTorch Onnx and CoreML.
MIT License
5.8k stars 965 forks source link

google.protobuf.message.DecodeError: Truncated message. #342

Open Enjia opened 6 years ago

Enjia commented 6 years ago

I have transformed my pytorch model to tf model, and when I started loading the saved_model.pb file:

from tensorflow.python.platform import gfile import tensorflow as tf import tensorboard

model = './saved_model.pb' graph = tf.get_default_graph() graph_def = graph.as_graph_def()

graph_def.ParseFromString(gfile.FastGFile(model, 'rb').read())

I got the following problem hint: google.protobuf.message.DecodeError: Truncated message.

I searched for some methods but they didn't work, BTW as far as I known it's an intractable problem. Would anybody give me some advice? Thanks!

namizzz commented 6 years ago

Hi @Enjia ,have you tried this way to load your tfmodel? It may work.

Enjia commented 6 years ago

@namizzz Thanks, it worked indeed. Meanwhile I checked the generated tf_resnet50.py file, I found the last two lines of the codes say:

ResNetnFinalLayernfcnnLinearnfcn442 = tf.layers.dense(ResNetnFinalLayernfcnnLinearnfcn442_flatten, 12, kernel_initializer = tf.constant_initializer(__weights_dict['ResNetnFinalLayernfcnnLinearnfcn442']['weights']), bias_initializer = tf.constant_initializer(__weights_dict['ResNetnFinalLayernfcnnLinearnfcn442']['bias']), use_bias = True) and return input, ResNetnFinalLayernfcnnLinearnfcn442 As the section of "Reuse the converted Tensorflow model" shows, how can I get the finally output (xxxxxxxxxfc442, what a expatiatory name) through operation "sess.graph.get_tensor_by_name('xxx:0')" or any other methods because it seems there's no name for last fc442 layer in graph. Thanks again!

namizzz commented 6 years ago

Hi @Enjia ,you can add name by yourself first then dump out the model file.

Enjia commented 6 years ago

@namizzz Thank you so much! I have tested my classification model and prediction score was identical to the original model.

Enjia commented 6 years ago

Hi @namizzz thanks for the solutions several days ago. Nowadays I am deploying my pb model to a webserver using Flask. The webserver will allow user to upload an image and then will generate a predictionmy. My thought is that if webserver has to load weights for each request and then do the inference, it will take a lot more time. So, I shall create the graph and load the weights and keep that in the memory so that I can quickly serve each incoming request. In consideration of this situation, I have to use the following code:

def load_graph(trained_model):   
    with tf.gfile.GFile(trained_model, "rb") as f:
        graph_def = tf.GraphDef()
        graph_def.ParseFromString(f.read())

    with tf.Graph().as_default() as graph:
        tf.import_graph_def(
            graph_def,
            input_map=None,
            return_elements=None,
            name=""
            )
    return graph

As I have mentioned some days ago, there's a problem with graph_def.ParseFromString(f.read()), that is "google.protobuf.message.DecodeError: Error parsing message"

The whole workflow code:

import sys
import os
import flask
from flask import render_template, send_from_directory, request, redirect,url_for
from werkzeug import secure_filename
from flask import jsonify
import base64
import StringIO
import tensorflow as tf 
import numpy as np
import cv2
app = flask.Flask(__name__)
UPLOAD_FOLDER='static'
def load_graph(trained_model):   
    with tf.gfile.GFile(trained_model, "rb") as f:
        graph_def = tf.GraphDef()
        graph_def.ParseFromString(f.read())
    with tf.Graph().as_default() as graph:
        tf.import_graph_def(
            graph_def,
            input_map=None,
            return_elements=None,
            name=""
            )
    return graph
@app.route('/')
def index():
    return "Webserver is running"
@app.route('/demo',methods=['POST','GET'])
def demo():
    if request.method == 'POST':
        upload_file = request.files['file']
        filename = secure_filename(upload_file.filename)
        upload_file.save(os.path.join(UPLOAD_FOLDER, filename))
        image_size=128
        num_channels=3
        images = []
        image = cv2.imread(os.path.join(UPLOAD_FOLDER, filename))
        image = cv2.resize(image, (image_size, image_size), cv2.INTER_LINEAR)
        images.append(image)
        images = np.array(images, dtype=np.uint8)
        images = images.astype('float32')
        images = np.multiply(images, 1.0/255.0)
        x_batch = images.reshape(1, image_size,image_size,num_channels)
        graph =app.graph
        y_pred = graph.get_tensor_by_name("y_pred:0")
        x= graph.get_tensor_by_name("x:0")
        y_test_images = np.zeros((1, 2))    
        sess= tf.Session(graph=graph)
        feed_dict_testing = {x: x_batch}
        result=sess.run(y_pred, feed_dict=feed_dict_testing)
        out={"cat":str(result[0][0]),"dog":str(result[0][1])}
        return jsonify(out)
   return '''
    <!doctype html>
    <html lang="en">
    <head>
      <title>Running my first AI Demo</title>
    </head>
    <body>
    <div class="site-wrapper">
        <div class="cover-container">
            <nav id="main">
                <a href="http://localhost:5000/demo" >HOME</a>
            </nav>
          <div class="inner cover">
          </div>
          <div class="mastfoot">
          <hr />
            <div class="container">
              <div style="margin-top:5%">
                    <h1 style="color:black">Dogs Cats Classification Demo</h1>
                    <h4 style="color:black">Upload new Image </h4>
                    <form method=post enctype=multipart/form-data>
                     <p><input type=file name=file>
                    <input type=submit style="color:black;" value=Upload>
                    </form>
                </div>  
            </div>
            </div>
     </div>
   </div>
</body>
</html>
    '''
app.graph=load_graph('./saved_model.pb')  
if __name__ == '__main__':
    app.run(host="0.0.0.0", port=int("5000"), debug=True, use_reloader=False)

Can you give me some advise to solve this protobuf issue or replace my load_graph function with other solutions so that avoid using sentences like "graph_def.ParseFromString(f.read())"? Thanks a lot!

namizzz commented 6 years ago

Hi @Enjia

def load_graph(trained_model):   
    with tf.gfile.GFile(trained_model, "rb") as f:
        graph_def = tf.GraphDef()
        graph_def.ParseFromString(f.read())

    with tf.Graph().as_default() as graph:
        tf.import_graph_def(
            graph_def,
            input_map=None,
            return_elements=None,
            name=""
            )
    return graph

It's a way to load fronzen protobuf TF model.

In MMdnn, we export TF model using API SavedModelBuilder.This is the right way to load the model:

export_dir = "./tf_resnet152"
with tf.Session(graph=tf.Graph()) as sess:
    tf.saved_model.loader.load(sess, [tf.saved_model.tag_constants.TRAINING], export_dir)

    x = sess.graph.get_tensor_by_name('input:0')
    y = sess.graph.get_tensor_by_name('xxxxxx:0')
    ......
    _y = sess.run(y, feed_dict={x: _x})

This two .pb are different.

If you want to save time of loading model saved by API SavedModelBuilder, it's better to ask in stackoverflow or tensorflow. Thanks!

Enjia commented 6 years ago

@namizzz I see, I will modify the code, thanks for your advice anyway!