V0.10.0 Release Plan

[cp5555]

Release Manager

@cp5555

Endgame

• [x] Code freeze: Dec. 8th, 2023
• [x] Bug Bash date: Dec. 11th, 2023
• [x] Release date: Dec. 25th, 2023

Main Features

SuperBench Improvement

1. • [x] Support Monitoring for AMD GPUs (#518 and #601)
2. • [x] Support ROCm5.7 and ROCm6.0 dockerfile (#587, #598, and #602)
3. • [x] Add MSCCL Support for Nvidia GPU (#584)
4. • [x] Fix NUMA Domains Swap Issue in NDv4 Topology File (#592)
5. • [x] Add ndv5 topo file (#597)
6. • [x] Fix NCCL and NCCL-test to 2.18.3 for hang issue in CUDA12.2 (#599)

Micro-benchmark Improvement

1. • [x] Add HPL random generator to gemm-flops with ROCm (#578)
2. • [x] Add DirectXGPURenderFPS Benchmark to measure the FPS of rendering simple frames (#549)
3. • [x] Add HWDecoderFPS Benchmark to measure the FPS of hardware decoder performance (#560)
4. • [x] Update Docker image for H100 support (#577)
5. • [x] Update MLC version into 3.10 for CUDA/ROCm dockerfile (#562)
6. • [x] Bug fix for GPU Burn test (#567)
7. • [x] Support INT8 in cublaslt function (#574)
8. • [x] Add hipBLASLt function benchmark (#576)
9. • [x] Support cpu-gpu and gpu-cpu in ib-validation (#581)
10. • [x] Support graph mode in NCCL/RCCL benchmarks for latency metrics (#583)
11. • [x] Support cpp implementation in distributed inference benchmark (#586 and #596)
12. • [x] Add O2 option for gpu_copy ROCm build (#589)
13. • [x] Support different hipblasLt data types in dist_inference (#590 and #603)
14. • [x] Support in-place in NCCL/RCCL benchmark (#591)
15. • [x] Support data type option in NCCL/RCCL benchmark (#595)
16. • [x] Improve P2P performance with fine-grained GPU memory in GPU-copy test for AMD GPUs (#593)
17. • [x] Update hipblaslt GEMM metric unit to tflops (#604)
18. • [x] Support FP8 for hipblaslt benchmark (#605)

Model Benchmark Improvement

1. • [x] Change torch.distributed.launch to torchrun (#556)
2. • [x] Support Megatron-LM/Megatron-Deepspeed GPT pretrain benchmark (#582 and #600)

Result Analysis

1. • [x] Support baseline generation from multiple nodes (#575)

Backlog

Micro-benchmark Improvement

1. Support cuDNN Backend API in cudnn-function.

Model Benchmark Improvement

1. Support VGG, LSTM, and GPT-2 small in TensorRT Inference Backend
2. Support VGG, LSTM, and GPT-2 small in ORT Inference Backend
3. Support more TensorRT parameters (Related to #366)