search

apache / tvm

Open deep learning compiler stack for cpu, gpu and specialized accelerators
https://tvm.apache.org/
Apache License 2.0
11.58k stars 3.43k forks source link

[RELEASE][DRAFT] TVM v0.6 Release candidate #4259

Closed tqchen closed 4 years ago

tqchen commented 4 years ago

Dear Community, thanks to everyone's effort in the past few months. This is a proposal to do a v0.6 release.

This release will be managed by the TVM PMC, with @yzhliu and myself as moderators. In the next few days we will be populating the release note in this thread. Most release note content will be derived from our monthly report

We also encourage everyone in the community to reply to the thread about pending PRs that should be included in the v0.6.

It is our first release after moving to the apache repo. So the main goal is about passing the general reviews to make sure the released product matches the ASF requirements. We hope that we can use this release to streamline the future releases

New Features

Relay in Production

Relay is a functional, differentiable programming language designed to be an expressive intermediate representation for machine learning systems. Relay supports algebraic data types, closures, control flow, and recursion, allowing it to directly represent more complex models than computation graph-based IRs (e.g., NNVM) can. In TVM v0.6, Relay is in stable phase and is ready for production.

Relay Virtual Machine

The Relay Virtual Machine (Relay VM) is the new generation of runtime to strike a balance between performance and flexibility when deploying and executing Relay programs. Previously, the graph runtime is able to utilize the fully static nature of the input graphs to perform aggressive optimization such as fully static allocation, and optimal memory reuse. When we introduce models which make use of control-flow, recursion, dynamic shapes, dynamic allocation we must change how execution works.

Relay VM is now usable and is able to achieve decent performance for a various of models and targets.

Training

Relay is designed to natively support first-order and higher-order differentiation. The automatic differentiation infrastructure is now usable and a count of operators with gradient support are available in v0.6 release.

Quantization

Low-bit inference is getting more and more popular as it benefits both the performance and storage usage. TVM now supports two types of quantization. 1. Automatic quantizaion takes floating-point precision model, does per-layer calibration and generates low-bit model. 2. TVM also imports pre-quantized model from Tensorflow and MXNet, a new dialect QNN is introduced to handle further lowering to normal operators.

Accelerator and Microcontroller Support

TSIM is introduced to improve software and hardware integration and simulation accuracy. It integrates the hardware development process into the software stack. TSIM enables VTA to provide a more accurate performance feedback, i.e. clock cycles, compared to the traditional functional model of a hardware accelerator. Moreover, Chisel implementation for VTA is availale and it runs on top of TSIM.

There has been a proliferation of resource-constrained and embedded devices that do not have operating systems or a mature software stack. MicroTVM is intended to support TVM on such bare-metal devices.

Rust Support

Rust language support in TVM includes two parts. 1. The frontend wraps the current C API and exposes a Rust programming model. 2. The backend serves as an alternative to C++ runtime. It privdes a standalone WASM module and security support, e.g., SGX.

Operator Support

Frontend and User Interface

Runtime and Backend Support

Language and Architecture

Feature Improvement

Symbolic shape enhancement

Language and Architecture

Arithmetic Analysis

Runtime and Backend Support

Frontend and User Interface

AutoTVM

Performance Improvements

Depreciations

yzhliu commented 4 years ago

Build and Test

Bug Fixes

Know Issue

FrozenGene commented 4 years ago

Though C++ RPC haven't been reviewed, personally, I wish C++ RPC https://github.com/apache/incubator-tvm/pull/4281 could enter into 0.6, which is a very useful feature in embedded devices, we almost use it in our production development every day.

yzhliu commented 4 years ago

I updated the issue, please let us know if anything's missing. (I didn't include "Build and Test" and "Bug fixes" section as the entire content happens to be long enough that github prevents me from posting it.)

One thing we need to discuss is whether NNVM needs to be deprecated in v0.6 release and how.

tqchen commented 4 years ago

wrt to nnvm. Our original proposal is to keep nnvm until v0.6 release. Which means nnvm will appear in v0.6 with limited support(the legacy api still passes test cases). Then we will remove nnvm compiler support after v0.6 release

yzhliu commented 4 years ago

@tqchen Shall we print warning message when ppl use nnvm?

tqchen commented 4 years ago

Yes, we can do that

u99127 commented 4 years ago

Should we have a known issues section ?

These are some initial thoughts from the top of my head over the last 5 minutes. I am sure that there are more.

In terms of 0.7 onwards I think we should be starting to think about criteria for a release and come up with some initial strawman proposals.

regards Ramana

yzhliu commented 4 years ago

@u99127 Good idea, will add.

yzhliu commented 4 years ago

Kindly remind that we would like to cut the release branch as well as the RC tag (RC0) this week. If you want to get any feature into the v0.6 release, please let us know. Once the release branch is cut, we no longer take new features. Bug fixes can still get in if necessary (but that means we need to cut another RC and go through the voting process again). @apache/tvm-committers

kevinthesun commented 4 years ago

Symbolic Shape Enhancement: Add more shape functions commonly used in cv models: https://github.com/apache/incubator-tvm/pull/4179

yzhliu commented 4 years ago

@kevinthesun Thanks. I'll go through the merging history make sure everything since the last monthly report gets into the release note.

masahi commented 4 years ago

We need to update benchmarks to use Relay, before we deprecate NNVM.

yzhliu commented 4 years ago

I don't have access to some of the devices, I can update the others. @merrymercy @tqchen what's your suggestion?

petrex commented 4 years ago

@yzhliu We'd love to push #4305 #4321 #4341 #4342 (fix and refactoring for AMD gpu backend) into TVM 0.6 release. thx

tqchen commented 4 years ago

@ZihengJiang can you help look into update the benchmark script?

petrex commented 4 years ago

@yzhliu @ZihengJiang I can provide some input for AMD gpu backend. please tag me in your PR. thanks

yzhliu commented 4 years ago

I updated the release notes to include recent changes, as well as some known issues as @u99127 suggested. please let me know if I missed anything.

vinx13 commented 4 years ago

typo: enhence -> enhance

yzhliu commented 4 years ago

@vinx13 thanks. updated.

yzhliu commented 4 years ago

folks, is there anything need to be merged? I plan to cut the release branch the end of the day.

vinx13 commented 4 years ago

@yzhliu let's get #4295 in, it's ready to merge once ci passed

tqchen commented 4 years ago

There is a temporary ci outrage due to http request timeout of the dataset let us make sure the ci is green. See also https://github.com/apache/incubator-tvm/issues/4378

yzhliu commented 4 years ago

@tqchen should we wait till #4378 is fixed?

tqchen commented 4 years ago

That should be a quick fix(see the reference PR which fixed a related problem. I would recommend doing so

junrushao commented 4 years ago

There are two weird characters, I am not sure if this is the issue of my browser:

1) After the sentence "Formalize Integer Arithmetic Analysis" 2) After the sentence "Support SpaceToBatchND/BatchToSpaceND in Tensorflow frontend".

FrozenGene commented 4 years ago

do we have the deadline now? because i have one pr of thread performance #4344

yzhliu commented 4 years ago

I have created v0.6.0 branch. I'm about to update the release notes.

yzhliu commented 4 years ago

Release note updated. Also remove the unexpected characters Thanks @junrushao1994

yzhliu commented 4 years ago

The next steps are roughly,

  1. Tag release candidate
  2. Create binary artifact
  3. Vote on dev@ mail-list (if fail, make change, go to step 1)
  4. Vote on general@ mail-list (if fail, make change, go to step 1)
  5. Finalize the release
tqchen commented 4 years ago

@yzhliu can we start a RFC thread before vote on dev@? just like a discuss thread that we can get feedbacks, before we open a vote, we can also invite ASF mentors to help check (after some thoughts, i think we could go with vote by checking the following items)

tqchen commented 4 years ago

FYI, the common checklist used for apache incubator release

yzhliu commented 4 years ago

@tqchen RFC after tag?

tqchen commented 4 years ago

sounds good

hsaputra commented 4 years ago

@tqchen , @yzhliu - please send request to review release candidate to dev@ list for TVM because not everyone probably subscribe or follow the Github repo list.

Before going to general@ list, let's vet and review the release candidate in the podling community to reduce the noise in the channel.

Thanks

tqchen commented 4 years ago

Thanks Henry This thread is already on dev@ :) — all github thread with RFC and DEV tags are automatically mirrored at dev@

tqchen commented 4 years ago

To further reduce the noise for our ASF PMC members and mentors, I would recommend future tag release related thread with [RELEASE] prefix, so our those who only want to watch related thread can setup email filter to watch this prefix

hsaputra commented 4 years ago

Thanks, @tqchen, Yes, I just noticed the emails also sent to dev@ list too :+1:

yzhliu commented 4 years ago

@hsaputra , I remember the release artifact needs to be uploaded to apache dist svn https://dist.apache.org/repos/dist/dev/incubator/ , is it still required? I don't see tvm under the repository, shall we create one?