TensorFlow Engine failed to run inference on TFHub model

roywei commented 3 years ago

Model:

https://tfhub.dev/google/nnlm-en-dim50/2

Error:

Exception in thread "main" org.tensorflow.exceptions.TFOutOfRangeException: Node 'NoOp' (type: 'NoOp', num of outputs: 0) does not have output 0
    at org.tensorflow.internal.c_api.AbstractTF_Status.throwExceptionIfNotOK(AbstractTF_Status.java:97)
    at org.tensorflow.Session.run(Session.java:666)
    at org.tensorflow.Session.access$100(Session.java:72)
    at org.tensorflow.Session$Runner.runHelper(Session.java:381)
    at org.tensorflow.Session$Runner.run(Session.java:329)
    at ai.djl.tensorflow.engine.TfSymbolBlock.forward(TfSymbolBlock.java:121)
    at ai.djl.nn.Block.forward(Block.java:117)
    at ai.djl.inference.Predictor.predict(Predictor.java:133)
    at ai.djl.inference.Predictor.batchPredict(Predictor.java:160)
    at ai.djl.inference.Predictor.predict(Predictor.java:128)

Steps to reproduce:

replace the model link to the above model in this example:

https://github.com/awslabs/djl/blob/master/examples/src/main/java/ai/djl/examples/inference/UniversalSentenceEncoder.java

roywei commented 3 years ago

Inspecting the model using saved model cli shows it takes no input, and the operator is NoOp, which is not expected The TFHub example works using python TF Hub API. Need to look into how it loads the model.

saved_model_cli show --dir ./  --tag_set serve --signature_def __saved_model_init_op                                                                                     ✘ INT  base

The given SavedModel SignatureDef contains the following input(s):
The given SavedModel SignatureDef contains the following output(s):
  outputs['__saved_model_init_op'] tensor_info:
      dtype: DT_INVALID
      shape: unknown_rank
      name: NoOp
Method name is:

roywei commented 3 years ago

The problem is with the Saved Model on TFHub, the signature keys were not properly exported.

Here is the workaround to load it back and export again, this model can now be loaded in DJL.

import tensorflow as tf
from tensorflow import keras
import tensorflow_hub as hub

hub_layer = hub.KerasLayer("https://tfhub.dev/google/nnlm-en-dim50/2",
                           input_shape=[], dtype=tf.string)

model = keras.Sequential()
model.add(hub_layer)
model.add(keras.layers.Dense(16, activation='relu'))
model.add(keras.layers.Dense(1, activation='sigmoid'))

model.summary()
tf.saved_model.save(model, "./converted/")

deepjavalibrary / djl