[TVM Integration] Improve TVM Integration: Tutorial + Investigating Numerical Issue - Githubissues

dmlc / gluon-nlp

NLP made easy

https://nlp.gluon.ai/

Apache License 2.0

2.55k stars 538 forks source link

[TVM Integration] Improve TVM Integration: Tutorial + Investigating Numerical Issue #1401

Open sxjscience opened 3 years ago

sxjscience commented 3 years ago

Description

TVM support has been added in https://github.com/dmlc/gluon-nlp/pull/1390. Also, we updated our benchmarking utility to support profiling the inference speed of TVM: https://github.com/dmlc/gluon-nlp/tree/master/scripts/benchmarks. We can further improve our document.

[ ] Attach the benchmark numbers. (Good for beginners)
[ ] Add a tutorial about how to convert GluonNLP models to TVM and do inference. (Good for beginners)
[ ] Investigate the numerical issues triggered when converting ALBERT to TVM. Currently, the test may still sometimes fail even if atol=1E-1, rtol=1E-3.

@dmlc/gluon-nlp-committers

szha commented 3 years ago

To publish the benchmark numbers we need to pick the environment. we've been using EC2 so whoever contributes this may require access to AWS and EC2.