[ROADMAP] TVM v0.8 Roadmap

[tqchen]

This roadmap for TVM v0.8. TVM is a community-driven project and we love your feedback and proposals on where we should be heading. Please open up discussion in the discussion forum as well as bring RFCs.

Feel free to volunteer yourself if you are interested in trying out some items(they do not have to be on the list).

Please also check out the help wanted list in the github issues on things that need help in the 0.8 cycle, we are going to focus on the following four areas. Which are summarized from the forum discussion.

We also welcome contributions along all the other areas, including more operator and model coverages, and they will be added to this list gradually. Please reply to this thread about things that are being worked on but missing.

We are looking at April, May timeframe for this release

Core Compiler

• [x] TensorIR scheduling support
• [x] New target system
• [x] While loop support
• [x] Consolidate compile engine pipeline to (IRModule -> IRModule)
• [ ] Type checker -- reusable forward shape analysis and type checking

Usability, Importers and Relay

• [ ] TVMC command line driver improvements
• [x] Windows compatibility improvements
• [ ] MLIR/HLO importer
• [ ] Improved quantization infra
• [ ] TorchScript interaction (importer and backend)
• [ ] Improve error messages and error categorization
• [x] Initial TVMScript support

Coverage

• [ ] Dynamic model and VM support improvements
• [ ] Sparse network support
• [ ] State support and basic block normal form
• [x] Gradient coverage for initial training workload support

Backends and Runtime

• [x] TensorRT BYOC support
• [x] Xilinx Vitis-AI
• [x] uTVM AOT runtime
• [x] uTVM library generator
• [x] Improve apple backend support

Automation

• [x] Stablize automatic scheduler
• [ ] Initial experimental autoscheduling for TensorIR
• [ ] Initial experimental auto-tensorization

[tqchen]

cc @apache/tvm-committers

[binarybana]

I personally would like to see a TVM 1.0 release following the 0.8 release. Where 0.8 is the last set of big features before we as a community focus on stabilization of core APIs, semantics, and testing processes for 1.0 hopefully landing towards the end of this year (2021).

Why am I mentioning this as part of the 0.8 roadmap? Because if the community wants to do 1.0 after 0.8, then it might change what features we want in 0.8 and when 0.8 occurs.

Also, a quick note: I'm not saying that experimental features wouldn't come out as part of TVM 1.0 or beyond, but they would be clearly marked as experimental or contrib and as part of the 1.0 release definition, we would have a clear stabilization process and criteria.

[tqchen]

Thanks @binarybana for bringing that up. I also agree that we should start to think about 1.0 around the end of the year. That could imply that we want to have a stronger focus on stablization, documentation in the later part. Would certainly be great to consider this possibility during planning.

[binarybana]

Also under coverage I'd like to see an entry for Leveraging ONNX test suite for more thorough testing and op support report generation.

[monklof]

Mentioned in https://discuss.tvm.apache.org/t/guideline-relay-aot/5977

We would like to see the “Fully Featured Relay AOT” feature in V0.8

Is there any plan on the “Fully Featured Relay AOT” Compiler?

We have a dynamic-shaped model, which is memory-intensive (the execution time of a single operator is short), the vm’s approach is hard to meet our performance requirements, the overhead is too large compared to the operators execution time.

So, we want a solution to minimize the overhead introduced by the extra code related to shape calculation and memory management.

AOT solution sounds like a good choice, since we can inline the computation related to shape calculation and eliminate the overhead related to interact with the VM.