Closed maybeLee closed 2 years ago
@gadagashwini I was able to replicate this issue on colab using TF v2.8.0, please refer to this gist.Thanks!
I looked at the code and could not see any obvious errors with the conversions. The problem does appear to only trigger when there is a SoftMax layer, with axis=-2
, and when converting to frozen graph.
A simple save and load leads the results that match the previous keras predictions:
model = keras.models.load_model("generated_model.h5")
h5_pred = model.predict(input)
np.sum(keras_pred - h5_pred) # 0
So the problem probably lies in the convert_variables_to_constants
(from tensorflow.python.framework.graph_util import convert_variables_to_constants
). I'd check with the TensorFlow tor TF2Onyx teams on this issue, since things work correctly from the Keras side of things.
A note: The TensorFlow APIs used in the colab are private or v1, i.e. no longer maintained (import_graph_def
, convert_variables_to_constant
). There is a convert_variables_to_constants_v2
which operates on tf.function
that you could look into.
System information.
Describe the problem. In short, I observe an output inconsistency between keras model's
predict
function and tensorflow'sprotobuf
and onnx's prediction result on a specific same model and same model weights.The model that trigger this issue is very simple (with only two layers)
The condition to trigger such inconsistency is to set the
axis
ofSoftmax
layer to be-2
:After generating such a model with randomly generated input, I find out that there is a large inconsistency between
keras_model.predict
, tensorflow'ssession.run
(yes I convert the model to a tensorflow protobuf using some code), andonnxruntime
's prediction.The prediction difference is as follows: Difference between TensorFlow (protobuf) and Keras: 300.0 Difference between TensorFlow (protobuf) and ONNXRuntime: 2.8684735e-07 Difference between Keras and ONNXRuntime: 300.0 Therefore, I suspect that this is an implementation bug inside Keras.
Below explains the detail of how I compare the result of different implementations (I paste all the implementation code below so this issue may be long......):
For the input, I always use an input tensor randomly generated by
numpy
:After generating this input, I will send the same input to all other implementations.
To get the keras' result, I use the most common way: construct a Keras model by
keras.models.Model
and use this constructed model to predict on the generated input and get keras result. I further save this model asgenerated_model.h5
so the weights to be loaded to other implementations will be exactly the same as keras's modelTo get the tensorflow (protobuf)'s result, I follow the post: https://medium.com/@johnsondsouza23/export-keras-model-to-protobuf-for-tensorflow-serving-101ad6c65142 to convert the
generated_model.h5
togenerated_model.pb
(tensorflow's protobuf format. You may access this colab link: https://colab.research.google.com/drive/1lHeZVtskThU8Baa7DFX0cuSyEoUu7PP6?usp=sharing to know how I did this.To get the prediction result on ONNXRuntime. I first use
tf2onnx
to convert the.h5
model to.onnx
model, I further use theonnxruntime
module to inference on the.onnx
model. Again, you may also follow the colab link to know how I did it.Describe the current behavior. Keras's h5 model output different than tensorflow's
protobuf
andonnx
Describe the expected behavior. These three outputs should be the same.
Contributing.