[RFC] Integrating the new MXNet website

[astonzhang]

About the new website

We heard of complaints and suggestions on the contents and UX of the MXNet website such as poor-quality tutorial examples and lack of feedback providing channel (e.g., https://github.com/apache/incubator-mxnet/issues/6493 and https://github.com/apache/incubator-mxnet/issues/12610). To make improvements on such, we propose to re-design the MXNet website.

The new website (WIP) will be published at http://beta.mxnet.io. We expect that the new website and the old website http://mxnet.incubator.apache.org/ will co-exist for some time until the old website can be completely replaced by the new website.

Proposed major changes

Below are proposed major new changes.

Design

1. We document each class or method in its own page to reduce the loading time. A TOC appears on the right hand side to allow users to navigate through different classes and methods.
2. We add disqus at the bottom of almost every page to allow users to comment and discuss.
3. We change the sphinx theme based on the material design, which has a modern look.

Content

1. We rely on the ecosystem of MXNet. Thus, we keep deep learning algorthm specific tutorials in Dive into Deep Learning, and keep toolkit-specific tutorials in the respective toolkit website/repo, such as GluonCV and GluonNLP.
2. For each programming laungage in the new website, we re-organize documentation into two directories: guide/ for tutorials and api/ for API references.
3. To improve quality of the documentation, we are also revising tutorials under guide/.
4. We highlight the use of Gluon.

Building process (learn more in Appendix A)

The new website (http://beta.mxnet.io) is currently built from https://github.com/mli/new-docs (hereinafter referred to as new-docs repo). As we will integrate the new-docs repo into the MXNet repo, the building process of the new website is subject to change (WIP).

1. We use conda to manage environments of dependencies.
2. Tutorials are written in markdown files. They will be converted to ipynb and get evaluated to generate outputs when the website is built. The dependencies for evaluating tutorials are specified in the conda environment.

If you are interested in more details about what's new in the new website, just check out this wiki written by @mli.

Status (will be updated)

1. The new website can be previewed at http://beta.mxnet.io.
2. Website redesign & wireframe are proposed at https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=103089084
3. We plan to open a feature branch to facilitate the integration of the new-docs repo.
4. To ensure high quality of contents, we plan to focus on a minimum viable product (MVP) in the initial release. The MVP may contain Python docs only. A proposed list of Python tutorials under guide/ in the initial release can be found in Appendix B. We will gradually add back other tutorials and docs of the other programming languages from the old website after the initial release.
5. We are revising contents of tutorials and open PRs can be found here. Feel free to review and comment these PRs.

Appendix A: Building process

In the following, we detail the building process at the new-docs repo. The building process of the new website is subject to change when we integrate the new-docs repo into the MXNet repo.

Overview

1. A pull request will trigger the execution of Jenkinsfile.
2. When Jenkinsfile is executed:
   1. Install all dependencies as specified in environment.yml, the environment is called mxnet-docs. The environment is managed by conda.
   2. Build Python docs in build/html (See details below)
   3. Build R docs in build/html/r
   4. If the pull request is merged to the master branch, it then executes build/upload_doc.sh to upload all files under build/html to s3://beta.mxnet.io

Building Python docs

• Instructions for Linux, Windows, and macOS Note: a fast EC2 instance is recommended. A GPU instance is required to test the tutorials, however a CPU instance is enough to build the site.

Building R docs (or another language)

The building process is similar to that of building Python docs.

Appendix B: Proposed list of Python tutorials in the MVP

Items below are usually the names of the existing files. For instance, 1-ndarray refers to https://github.com/mli/new-docs/blob/master/python/guide/crash-course/1-ndarray.md. If such a file does not exist, such as Symbol, we will create it.

• Getting Started

  • A 60-minute crash course

  • 1-ndarray

  • 2-nn

  • 3-autograd

  • 4-train

  • 5-predict

  • 6-use_gpus

  • Moving from other DL frameworks

  • to-mxnet/pytorch.html

  • to-mxnet/tf.html

• Packages and Modules

  • ndarray
  • ndarray (need more details, refer to https://en.d2l.ai/chapter_crashcourse/ndarray.html)
  • sparse
  • autograd (need more details)
  • Using GPU (refer to https://en.d2l.ai/chapter_deep-learning-computation/use-gpu.html)
  • optimizer
  • lr_scheduler
  • Symbol
  • gluon
  • nn (need more details, refer to https://en.d2l.ai/chapter_deep-learning-computation/model-construction.html)
  • parameters (need more details, refer to https://en.d2l.ai/chapter_deep-learning-computation/parameters.html)
  • init (need more details, refer to https://en.d2l.ai/chapter_deep-learning-computation/parameters.html and https://en.d2l.ai/chapter_deep-learning-computation/deferred-init.html)
  • custom-layer (need more details, refer to https://en.d2l.ai/chapter_deep-learning-computation/custom-layer.html)
  • data
  • image_augmentation (should show how to use gluon.data.vision.transforms.Compose, refer to https://en.d2l.ai/chapter_computer-vision/image-augmentation.html)
  • file I/O (refer to https://en.d2l.ai/chapter_deep-learning-computation/read-write.html)
  • loss (should explain how to create customized loss)
  • trainer
  • hybridize (need more details, refer to https://en.d2l.ai/chapter_computational-performance/hybridize.html)
  • model_zoo

• Performance

  • Essentials
  • (optional) perf
  • profiler
  • Compression
  • float16
  • (optional) int8
  • Accelerated Backend
  • (optional) tensor-rt

• Deployment

  • Deploy into Another Language
  • cpp
  • scala/java
  • ONNX

• Customization (extend)

  • customop (should address customer complaints: https://discuss.gluon.ai/t/topic/10822)

[mxnet-label-bot]

Hey, this is the MXNet Label Bot. Thank you for submitting the issue! I will try and suggest some labels so that the appropriate MXNet community members can help resolve it. Here are my recommended labels: Doc

[astonzhang]

@mxnet-label-bot Add [RFC]

[aaronmarkham]

I think there's some discussion to be had around how the APIs drill down. I have a proposed flow in the wiki than now fleshing things out a bit more. I think this flow could remove the need for a /guide folder as these are all under /api and the elements of "guide" are interwoven in every page. https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=103089084

I've taken the tact following this pattern: List of language APIs, API landing page + related materials, API topic (like data but talk about images and text), [API subtopic (like vision having models), API children (until the tree is exhausted)]. The last two tend to get intermingled because some branches are shorter than others.

The other aspect that I'm presenting is using a use case flow for presenting the APIs as opposed to just having a clusters or collections of APIs without any apparent order or relation. Once you get to the last layer of a list of functions then these would be categorical then alphabetical. Given that there is much overlap in the APIs there would be some duplication for things like NDArray or Symbol. However, a Gluon user wouldn't really see this, they'd just assume all of the available APIs presented in their guide are theirs to use. Conversely, a user diving into the Symbol API would see all relevant APIs for them but would never see any mention of gluon. Common packages like NDArray would be described in both guides. Examples would take care not to confuse the reader with inappropriate mixing of patterns. That's my goal at least.

Please make comments on the UX wireframes directly! https://gallery.io/projects/MCHbtQVoQ2HCZfrqGyueCUJz/files/MCEJu8Y2hyDScXaVWHecaLCwYpWsjC1kkZY

[aaronmarkham]

While not specifically addressed in the RFC, I'd like to recommend that we do not build multiple versions of the site in the MVP. Once an architecture is established, then the multiple language bindings, and some kind of actual UX direction, and then we can look versions and how best to provide them as a "nice-to-have". Otherwise, I think giving out directions on how to build a specific version of the docs should be sufficient.

[szha]

This change has been accepted to the mxnet website. Thanks to those who helped on making it happen!