search

iree-org / iree

A retargetable MLIR-based machine learning compiler and runtime toolkit.
http://iree.dev/
Apache License 2.0
2.59k stars 581 forks source link

Teach `iree-tf-import` more smarts regarding SavedModel handling #5556

Open antiagainst opened 3 years ago

antiagainst commented 3 years ago

Discovered a few sharp corners when refreshing IREE docs; listing them here so @silvasean can do magic. :)

1) Right now iree-tf-import requires manually specifying the SavedModel version. If the wrong version is given, then an empty module op will be generated without any hints. This is not quite user friendly. We probably should automatically deduce the versions there; at least give the user an hint if the output is just an empty module op.

1) SavedModels may or may not contain serving signatures / entry points for IREE compilation flow. If we can have a way to dump all available serving signatures to verify that without requiring loading the SavedModel using Python it would be nice. If there are no serving signatures, then giving links to the user to help them re-export could save quite some head scratching time.

claforte commented 2 years ago

Bumped into this issue trying to import a TF ResNet50 model. This is very frustrating and discouraging for new users like me, who know Pytorch but aren't familiar with TF, yet want to evaluate IREE's full performance (which isn't yet possible with pytorch models). IMHO the documented workflow (https://iree-org.github.io/iree/getting-started/tensorflow/) is too brittle and doesn't offer any hints on how to debug such problems.

ScottTodd commented 2 years ago

Thanks for the feedback, I feel your frustration too :(

This sample (mentioned on this issue also) shows a bit of how to debug issues importing TF programs: https://colab.research.google.com/github/iree-org/iree/blob/main/samples/colab/tensorflow_hub_import.ipynb, and I agree that the entire workflow is brittle. Some of that can be improved with documentation and smarter tools.

The TensorFlow Lite path is more stable: https://iree-org.github.io/iree/getting-started/tflite/, and I'd recommend it for common model architectures like ResNet50.