Refactoring NNI python package and code hierarchy

[liuzhe-lz]

Original code hierarchy of NNI is designed under the assumption that it would be a multi-language framework, which is no long the case. To make NNI easier to maintain as a Python framework, I propose to refactor the code hierarchy in v2.0 release. This issue tracks the design of new hierarchy.

[liuzhe-lz]

In new design, NNI will be organized as one Python package, and TypeScript modules will be treated as its resource files.

So the top-level hierarchy will be look like this:

• docs
• examples
• python source
• typescript source
• python unit test
• integration test
• setup.py and requirements.txt
• Dockerfile
• (optional) install scripts which simple invoke setup.py
• README, LICENSE, and repo config files (.gitignore, azure-pipelines.yml, etc)

Inside python source directory, codes are grouped to following categories:

• algorithms
• AutoML SDK: algorithm interface and trial (model) interface
• runtime: underlying logic to integrate with NNI manager and training service
• utility modules mainly for internal use (algorithm package, GPU metrics, annotation parser, etc)
• nnictl

AutoML SDK should first be grouped by features (HPO, NAS, compression, feature engineering), and then by framework.

Despite the file hierarchy, some modules should be able to import from top level:

• main classes of built-in algorithms
• trial API
• (optional) major base classes of algorithm API (Tuner, Mutator, Pruner, etc)
• anything required to keep backward-compatibility

[liuzhe-lz]

Classification of source files:

meta        .github/
>           deployment/
>               docker/
install             Dockerfile
doc                 README.md
doc                 README_zh_CN.md
install         pypi/
install         deployment-pipelines.yml
doc         docs/
example     examples/
>           src/
ts              nasui/
ts              nni_manager/
>               sdk/
>                   pycli/
>                       nnictl/
-                           __init__.py
util                        nni_client.py
install                 setup.py
>                   pynni/
>                       nni/
algorithm                   batch_tuner/
algorithm                   bohb_advisor/
>                           compression/
-                               tensorflow/
>                               torch/
algorithm*                          pruning/
>                                   quantization/
-                                       __init__.py
algorithm                               quantizers.py
>                                   speedup/
-                                       __init__.py
?                                       compress_modules.py
?                                       compressor.py
?                                       infer_shape.py
>                                   utils/
-                                       __init__.py
?                                       config_validation.py
?                                       counter.py
?                                       mask_conflict.py
?                                       num_param_counter.py
?                                       sensitivity_analysis.py
?                                       shape_dependency.py
-                                   __init__.py
library                             compressor.py
library                             default_layers.py
-                               __init__.py
algorithm                   curvefitting_assessor/
algorithm                   evolution_tuner/
>                           feature_engineering/
algorithm                       gbdt_selector/
algorithm                       gradient_selector/
-                               __init__.py
library                         feature_selector.py
algorithm                   gp_tuner/
algorithm                   gridsearch_tuner/
algorithm                   hyperband_advisor/
algorithm                   hyperopt_tuner/
algorithm                   medianstop_assessor/
algorithm                   metis_tuner/
>                           nas/
?                               benchmarks/
>                               pytorch/
algorithm                           cdarts/
algorithm                           classic_nas/
algorithm                           darts/
algorithm                           enas/
algorithm                           pdarts/
algorithm                           proxylessnas/
algorithm                           random/
?                                   search_space_zoo/
algorithm                           spos/
-                                   __init__.py
library                             base_mutator.py
library                             base_trainer.py
library                             callbacks.py
library                             fixed.py
library                             mutables.py
library                             mutator.py
library                             trainer.py
library                             utils.py
-                               tensorflow/
-                               __init__.py
algorithm                   networkmorphism_tuner/
algorithm                   pbt_tuner/
runtime                     platform/
algorithm                   ppo_tuner/
algorithm                   smac_tuner/
-                           __init__.py
runtime                     __main__.py
library                     _graph_utils.py
library                     assessor.py
?                           common.py
?                           constants.py
runtime                     env_vars.py
runtime*                    msg_dispatcher_base.py
runtime*                    msg_dispatcher.py
?                           nas_utils.py
util                        package_utils.py
library                     parameter_expressions.py
runtime                     protocol.py
remove                      recoverable.py
library                     smartparam.py
library                     trial.py
library                     tuner.py
?                           utils.py
-                       pylintrc
remove                  README.md
remove                  README_zh_CN.md
install                 requirements.txt
install                 setup.py
test                    tests
test                    ut.sh
ts              webui/
test        test/
>           tools/
>               nni_annotation/
example             examples/
test                testcase/
doc                 README.md
doc                 README_zh_CN.md
-                   __init__.py
util                code_generator.py
util                search_space_generator.py
util                specific_code_generator.py
util                test_annotation.py
?               nni_cmd/
>               nni_gpu_tool/
-                   __init__.py
util                gpu_metrics_collector.py
?               nni_trial_tool/
doc             README.md
doc             README_zh_CN.md
remove          bash-completion
install         setup.py
meta        .gitattributes
meta        .gitignore
meta        LICENSE
install     Makefile
meta        README.md
meta        README_zh_CN.md
remove      README.Makefile.md
remove      README.Makefile_zh_CN.md
remove      _config.yml
meta        azure-pipelines.yml
meta        crowdin.yml
install     install.ps1
install     install.sh
-           pylintrc
install     setup.py
install     uninstall.ps1