search

onnx / tensorflow-onnx

Convert TensorFlow, Keras, Tensorflow.js and Tflite models to ONNX
Apache License 2.0
2.33k stars 432 forks source link

Support for `LayerNormalization` from ONNX opset 17 #2183

Open pwuertz opened 1 year ago

pwuertz commented 1 year ago

When converting a tensorflow.keras.layers.LayerNormalization layer to ONNX, tf2onnx currently decomposes layer normalizations into rather complex subgraphs with batch norms and more basic building blocks. Inference engines (like TensorRT in the following example) are hardly able to deduce the original layer norm op from the graph and will have to follow the instructions to the letter: image

Since ONNX version 17 however, tensorflow.keras.layers.LayerNormalization layers are directly convertible to ONNX LayerNormalization operators. This of course produces a much simpler and expressive ONNX graph and leaves more room for the inference engine for possible optimizations (ONNX-to-TensorRT example again): image

The current version of tf2onnx doesn't seem to produce those new LayerNormalization operators yet, at least not with tf2onnx.convert.from_keras(model, opset=17).

DayDayupupupup commented 1 year ago

Same issue. Is there any plan to support rewrite LayerNormalization（opset 17）when use tf‘s savedmodel format？

charvey2718 commented 1 year ago

Same issue for me.

ShuaiShao93 commented 1 year ago

I have a tentative fix in https://github.com/onnx/tensorflow-onnx/pull/2250. Feel free to take it over