search

hpcaitech / ColossalAI-Examples

Examples of training models with hybrid parallelism using ColossalAI
Apache License 2.0
334 stars 102 forks source link

Added BigGAN example #90

Closed ExtremeViscent closed 2 years ago

ExtremeViscent commented 2 years ago

A Colossal-AI implementation of Large Scale GAN Training for High Fidelity Natural Image Synthesis. Data and multiple tensor parallel mode supported.

binmakeswell commented 2 years ago

Hi @ExtremeViscent Thanks for your contribution! Here are some suggestions for improvement:

  1. Organize files, there are many files in the root directory.
  2. Polish README: What is the expectation training result for this example, and did you verify its correctness (please provide the satisfactory log)?
  3. Currently, this example seems only support 2d tensor parallel. Do you consider data/other tensor/pipeline parallel, ZeRO, etc.? Could you please try to support them, provide the configs, and update expected results in README? Thanks.
binmakeswell commented 2 years ago

This PR is closed due to inactivity.