tensorflow / tensorrt

TensorFlow/TensorRT integration
Apache License 2.0
736 stars 226 forks source link

No speed improvements after TF-TRT optimizing on a tensorflow BERT model #330

Open SohaKhazaeli opened 2 years ago

SohaKhazaeli commented 2 years ago

After optimizing the model with either FP32 or FP16 I don't get any speed improvements.

The optimization is done on tensorflow/tensorflow:2.10.0-gpu docker image. The model uses tensorflow-text and tf-models-official libraries

This is the log from optimization process:

image