search

hpcaitech / ColossalAI-Examples

Examples of training models with hybrid parallelism using ColossalAI
Apache License 2.0
334 stars 102 forks source link

mlpmixer #80

Closed fanjinfucool closed 2 years ago

fanjinfucool commented 2 years ago

MLPMIXER

binmakeswell commented 2 years ago

Hi @fanjinfucool Currently, this example does not support main features in Colossal-AI, eg. data/tensor/pipeline parallel, ZeRO, etc. Could you please try to support them, provide the configs, and update expected results in README? Thanks.