search

hpcaitech / ColossalAI-Examples

Examples of training models with hybrid parallelism using ColossalAI
Apache License 2.0
334 stars 102 forks source link

Biology implement of ColossalAI #82

Closed XIXIYOUNG2018 closed 2 years ago

binmakeswell commented 2 years ago

Hi @XIXIYOUNG2018 Thanks for your contribution! Here are some suggestions for improvement:

  1. Upload the data file to Google Drive or Baiduwangpan, rather than here.
  2. Polish README: Why did you implement this biology example, how this example will help others, what is the expectation training result for this example, and did you verify its correctness (please provide the satisfactory log)?
  3. Is there any reason for you to use resnet18 in this example? Colossal-AI is mainly designed for large AI models with parallel techniques, which perform better for transformer structures.
  4. Currently, this example does not support main features in Colossal-AI, eg. data/tensor/pipeline/sequence parallel, ZeRO, etc. Is there any reason why this example is limited to a single GPU?
  5. Rebase your commit into one.
binmakeswell commented 2 years ago

This PR is closed due to inactivity.