cherry pick: Deploy BERT model - Script (#1237)

[MoisesHer]

• Add example script to deploy BERT

• Add options to better measure performance

• Allow specification of path for exported model

• Add option to use custom graph pass

• Add optimization for MHA in custom graph pass

• Correct bug with input shapes in optimize_for

• correct typo

• fix lint

• fix lint

• Add documentation

• Add documentation for using deploy script

• Correct typo/add spaces in documentation

• Add setup.py to compile pass, update documentation

• Fix bug in path to include dir & fix pylint

• Add unitest for deploy bert script

• change CUDA version in wheel

• test latest wheel

• change path to custom pass library

• fixing trigger custom pass compilation

• fix lint

• fix lint

• Update mxnet pip version

• Only GPU versions changed

• fix lint

• change wheel to include mkl headers

• lint docstring

• remove debug print

• change include paths

• lint

• debugging lib_api.h

• debugging lib_api.h

• debugging

• Disable test for now

• skip test if mxnet_version < 1.7.0

• use pytest.mark.skipif to skip test

• test only BERT-base (fp16/fp32, SST/QA, embeddings) to avoid timeout

Co-authored-by: Leonard Lausen lausen@amazon.com

Description

(Brief description on what this PR is about)

Checklist

Essentials

• [ ] PR's title starts with a category (e.g. [BUGFIX], [MODEL], [TUTORIAL], [FEATURE], [DOC], etc)
• [ ] Changes are complete (i.e. I finished coding on this PR)
• [ ] All changes have test coverage
• [ ] Code is well-documented

Changes

• [ ] Feature1, tests, (and when applicable, API doc)
• [ ] Feature2, tests, (and when applicable, API doc)

Comments

• If this change is a backward incompatible change, why must this change be made.
• Interesting edge cases to note here

cc @dmlc/gluon-nlp-team

[mli]

Job PR-1357/1 is complete. Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-1357/1/index.html