NNFusion v0.3 (JAN~FEB) Release Plan

[AlisaChen98]

This is the draft plan for the upcoming iteration in Jan ~ Feb, it's a 6 week's iteration.

Release Plan for V0.3

• Release Manager: @wenxcs
• Feature Freeze Date: 2021.~03.04~ TBD
• Code Freeze Date: 5.18 with fixes for #266, #265, #264, #263
• Test Verification/Bug Bash: 5.18 - 5.19
• Release Date: 5.20

• Release Note:

  Annotation Legend:

Mark	Description
🏃‍♂️	work in progress
👪	work under review
✋	blocked task

Feature

Mechanism

• [x] P0 - Custom op support | @jlxue @wenxcs @mzmssg | Testing @wenxcs

  • [x] C++ | 3 days
  • [x] JSON | 2 days
  • ~TF Protobuf~
  • ~ONNX~
  • [x] doc @jlxue

• [x] P0 - Support training | Testing @xysmlx

  • [x] bert training on real dateset ｜2 days｜@mzmssg
  • [x] LSTM Pytorch @xysmlx
  • [x] doc @mzmssg

• [x] P0 - 🏃‍♂️ C++ support HLSL | @jlxue | | Testing @mzmssg

  • [x] port wrapper lib to XDev ｜2 days｜ @jlxue
  • [x] C++ codegen for HLSL｜1 day｜ @xiayuqing0622

Refactor/Improvement

• [x] P0 - Bert Related Optimization | 14～18days｜@xiayuqing0622 | Testing @yuanjie
  • [x] LayerNorm fusion | 2~3 days
  • [x] Embedding layer fusion| 2~3 days
  • [x] Skip layer norm fusion| 2~3 days
  • [x] Attention fusion| 2~3 days
  • [x] Gelu fusion| 2~3 days
  • [x] Matmul add fusion| 2~3 days

Bug Fix

• model debug

  • [x] check grad number & Unsortedsegmentsum op: @yuanjie

• Antares translate IR bugs: (check with Yuxiao): @jlxue @weicui

  • [x] padding for AvgPool #172
  • [x] DepthwiseConv2d #192 (asymmetric padding)

• [x] Generate valid tensor name when the given name is invalid #163 @xysmlx

• [x] onnx op_bridge check #218 @mzmssg

---

Deferred

• offline inference | @wenxcs

  • [ ] bert hugging face | 5 days

    • [ ] autotunning for new hardware
    • [ ] support apex

  • [ ] nnstream: streaming offline inference interface| 10 days

  • 🏃‍♂️ general sub-graph substitution | @Niupple

  • [x] matching and substituting algorithms | 4 days

  • [ ] python rule editing interface | 2 days

• [x] support HLSL end-to-end python interface | @mzmssg

• support low-precision & mixed-precision | @Niupple

  • [ ] train model with pure half precision | 3 days
  • [ ] fp32 master copy of weights | 3 days
  • [ ] manual/auto loss scaling factor | 3 days

• Control-flow support ｜@xysmlx

  • [ ] control-flow op define: loop, if｜@xysmlx
  • [ ] onnx frontend for control-flow ops｜@xysmlx
  • [ ] codegen graph pass whose input is a graph and output is a function｜@xysmlx
  • [ ] codegen for control-flow ops ｜@xysmlx
  • [ ] refactor codegen & other graph passes ｜@xysmlx @xiayuqing0622

• sub-graph substitution | @wenxcs @Niupple #127

• Test strategy ｜ @wenxcs

• support pip install ｜10 days | @mzmssg

  • [x] refine python code structure
  • [x] pip install python sources
  • [ ] ~pip install nnfusion cli~

---

Stretch Goals

• docker image support low version of cuda driver | @wenxcs
• Control-flow support ｜@xysmlx
  • [ ] python integration
• new profiler: support different profiling information, e.g., time, kernel time provided by nvprof | @xysmlx
  • [ ] Base profiler | 5 days
  • [ ] Update existing profilers | 1 day
  • [ ] CUDA profiler: kernel time, num of __syncthreads(), num of active thread blocks, shared memory, grid/block size | 2 days
  • [ ] Kernel profiling pass | 1-2 days
• Profiling cache DB | @xysmlx
  • [ ] DB schema and interfaces
  • [ ] Integrate with profiler
• BlockFusion | @xysmlx
  • [ ] BlockFusion active thread block check #50
  • [ ] Process __shared and syncthreads() with profiler to support any BlockCudaEmitter::get_kernel_cache_entry() function
• Generate identifier to identify unique op configs | @xysmlx

[nnfbot]

Thanks for the report @AlisaChen98! I will look into it ASAP! (I'm a bot).

[AlisaChen98]

Our main target of this release:

1. Custom op support
2. Offline inference(Take Bert as a typical example)
3. Other fix and enhancements

[wenxcs]

NNFusion v0.3版本发布

特性更新

• 支持在真实数据集上端到端的BERT模型训练（ONNX格式）
• 增加一系列针对Transformer模型的算子融合优化
• 提供扩展自定义算子的C++和JSON接口
• 支持新的HLSL代码生成

  其他更新

• 更新相关文件
• 修复bug