[Tracking Issue] [Experimental] Relax Fallback Mechanism for Missing Operators

[sunggg]

Operator coverage has been one of the biggest problems. Currently, when certain operator is not supported in TVM, importer simply throws an error and there is no way to proceed until the missing operator is implemented. To overcome this issue, we came up with an idea to offload those missing operators to the frontend runtime and run them together with the rest of TVM operators. For example, if there is an missing operator in PyTorch importer, we can run it on the PyTorch runtime and interact with the rest of the model.

My branch here demonstrates this idea with TorchFx frontend and Libtorch runtime and tested with TorchVision models.

Progress Tracking

• [x] P1. Introduce TorchFx frontend with minimal operators: https://github.com/tlc-pack/relax/pull/419
• [ ] P2. Implement fallback mechanism
• [ ] P3. Demo at the community meeting and collect feedback
• [ ] P4. Extend TorchFx frontend to support all existing operators

Future Possibilites

• Current fallback mechanism is implemented at high-level by using Libtorch C++ frontend API for fast prototyping. Depending on the potential issues that we may face in the future (e.g., performance overhead), we may consider implementing the similar mechanism at the lower-level with Aten-level API.
• Currently, TorchFx seems to require special handling for Bert models (issue). Find out more organic way to handle Bert models.

cc. @junrushao @YuchenJin @tqchen

[Hzfengsy]

I'm suffering from the same issue with Bert models.