search

microsoft / superbenchmark

A validation and profiling tool for AI infrastructure
https://aka.ms/superbench
MIT License
248 stars 55 forks source link

V0.11.0 Release Plan #617

Open cp5555 opened 5 months ago

cp5555 commented 5 months ago

Release Manager

@cp5555

Endgame

Main Features

SuperBench Improvement

    • [x] Add CUDA 12.4 dockerfile (#619)
    • [x] Improve document (#628 and #632)
    • [x] Update omegaconf version to 2.3.0 (#631)
    • [x] Fix MSCCL build error in CUDA12.4 docker build pipeline (#633)
    • [x] Update Docker Exec Command for Persistent HPCX Environment (#635)
    • [x] Use types-setuptools to replace types-pkg_resources (#637)
    • [x] Update Docker Exec Command for Persistent HPCX Environment (#635)
    • [x] Fix bug of failure test and warning of pandas in data diagnosis (#638)
    • [ ] Remove pytest version constraint (<=7.4.4)

Micro-benchmark Improvement

    • [x] Add hipblasLt tuning to dist-inference cpp implementation (#616)
    • [ ] Add support for NVIDIA L4/L40/L40s GPUs in gemm-flops (#634)
    • [x] Upgrade mlc to v3.11 (#620)
    • [ ] Support cuDNN Backend API in cudnn-function.

Model Benchmark Improvement

  1. Support VGG, LSTM, and GPT-2 small in TensorRT Inference Backend
  2. Support VGG, LSTM, and GPT-2 small in ORT Inference Backend
  3. Support more TensorRT parameters (Related to #366)

Result Analysis