Inference not working with ModelServer, but is working with saved_model_cli

My team and I worked on a text classification model using an LSTM and saved it using the saved_model_builder. We then tested it using the saved_model_cli, and inference was working. However, when we ran the ModelServer and called it with a gRPC call, it is giving us the following error:

Traceback (most recent call last):
    File "analyst_rating_client.py", line 91, in post
        response = stub.Predict(request,1.0)
    File "/usr/local/lib/python2.7/dist-packages/grpc/beta/_client_adaptations.py", line 309, in __call__
        self._request_serializer, self._response_deserializer)
    File "/usr/local/lib/python2.7/dist-packages/grpc/beta/_client_adaptations.py", line 195, in _blocking_unary_unary
        raise _abortion_error(rpc_error_call)
AbortionError: AbortionError(code=StatusCode.FAILED_PRECONDITION, details="Attempting to use uninitialized value fully_connected/biases
    [[Node: fully_connected/biases/read = Identity[T=DT_FLOAT, _class=["loc:@fully_connected/biases"], _output_shapes=[[1]], _device="/job:localhost/replica:0/task:0/cpu:0"](fully_connected/biases)]]")

We are not sure why this variable is not being initialized in this case, but it is working fine with saved_model_cli. Has anyone faced this issue?

Here is our saved_model script:

def lstm_cell():
    lstm = tf.contrib.rnn.BasicLSTMCell(lstm_size, reuse=tf.get_variable_scope().reuse)
    return tf.contrib.rnn.DropoutWrapper(lstm, output_keep_prob=keep_prob)

tf.reset_default_graph()
with tf.name_scope('inputs'):
    inputs_ = tf.placeholder(tf.int32, [None, None], name="input_ints")
    labels_ = tf.placeholder(tf.int32, [None, None], name="labels")
with tf.name_scope("Embeddings"):
    embedding = tf.Variable(tf.random_uniform((vocab_size, embed_size), -1, 1))
    embed = tf.nn.embedding_lookup(embedding, inputs_)
with tf.name_scope("RNN_layers"):
    cell = tf.contrib.rnn.MultiRNNCell([lstm_cell() for _ in range(lstm_layers)])
initial_state = cell.zero_state(batch_size, tf.float32)
with tf.name_scope("RNN_forward"):
    outputs, final_state = tf.nn.dynamic_rnn(cell, embed, initial_state=initial_state)
with tf.name_scope('predictions'):
    predictions = tf.contrib.layers.fully_connected(outputs[:, -1], 1, activation_fn=tf.sigmoid)

saver = tf.train.Saver()
with tf.Session() as sess:
    saver.restore(sess, import_path)
    builder = saved_model_builder.SavedModelBuilder(export_path)

    prediction_inputs = utils.build_tensor_info(inputs_)
    prediction_outputs = utils.build_tensor_info(predictions)

    prediction_signature = (tf.saved_model.signature_def_utils.build_signature_def(
        inputs={signature_constants.PREDICT_INPUTS: prediction_inputs},
        outputs={signature_constants.PREDICT_OUTPUTS: prediction_outputs},
        method_name=tf.saved_model.signature_constants.PREDICT_METHOD_NAME))

    legacy_init_op = tf.group(tf.tables_initializer(), name='legacy_init_op')

    builder.add_meta_graph_and_variables(
        sess, [tag_constants.SERVING],
        signature_def_map={
            signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY:
                prediction_signature
        },
        legacy_init_op=legacy_init_op)

    builder.save()

Here is some information about our saved model:

$ saved_model_cli show --dir ./python2_prediction_SavedModelFinalInputInts --all

MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:

signature_def['serving_default']:
The given SavedModel SignatureDef contains the following input(s):
inputs['inputs'] tensor_info:
    dtype: DT_INT32
    shape: (-1, -1)
    name: inputs/input_ints:0
The given SavedModel SignatureDef contains the following output(s):
outputs['outputs'] tensor_info:
    dtype: DT_FLOAT
    shape: (1, 1)
    name: predictions/fully_connected/Sigmoid:0
Method name is: tensorflow/serving/predict

Here is our gRPC client request:

    channel = implementations.insecure_channel('localhost',9000)
    stub = prediction_service_pb2.beta_create_PredictionService_stub(channel)
    request = predict_pb2.PredictRequest()
    request.model_spec.name = 'default'
    request.model_spec.signature_name = signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY
    # input_nums is a list of list of integers (of shape [1, 100])
    request.inputs[signature_constants.CLASSIFY_INPUTS].CopyFrom(tf.contrib.util.make_tensor_proto(input_nums, shape=[1,100],dtype=tf.int32))
    response = stub.Predict(request,10.0)

Thanks in advance!

tensorflow / serving

Inference not working with ModelServer, but is working with saved_model_cli #674