Enhance TorchBench coverage for large distributed workloads with CI support for IBM Cloud

pytorch / benchmark

TorchBench is a collection of open source benchmarks used to evaluate PyTorch performance.

BSD 3-Clause "New" or "Revised" License

837 stars 269 forks source link

Open spzala opened 1 year ago

spzala commented 1 year ago

The proposed work tasks are as below:

[ ] Enable CI support for IBM Cloud to enhance the testing infrastructure for FSDP
[ ] Benchmark new model(s) for FSDP training - e.g. add new hf_T5 with 3B parameters, etc.

xuzhao9 commented 1 year ago

Looping @mrshenli into the discussion, since the PyTorch Distributed team is also interested in building benchmarks in distributed setup.