search

hpcaitech / ColossalAI-Examples

Examples of training models with hybrid parallelism using ColossalAI
Apache License 2.0
334 stars 102 forks source link

[titans]use titans to support deepnet. #112

Closed YuliangLiu0306 closed 2 years ago

YuliangLiu0306 commented 2 years ago
  1. Use model from titans to support deepnet training.
  2. Fix a bug caused by checkpoint.
  3. Fix some bugs due to using the outdated api.
  4. Non-pipeline version deepnet don't build in origin examples, so I may build a non-pipeline model and use pipelinable to instead the pipeline version deepnet in other PR.