search

hpcaitech / ColossalAI-Examples

Examples of training models with hybrid parallelism using ColossalAI
Apache License 2.0
334 stars 102 forks source link

add bert examples #47

Closed Wesley-Jzy closed 2 years ago

FrankLeeeee commented 2 years ago

Hi @Wesley-Jzy , can you provide a detailed readme instead of an empty one?

Wesley-Jzy commented 2 years ago

@FrankLeeeee Thanks for review. It's still on the way since I try to fathom how to change model for TP and PP features now. I will add detailed README and delete code duplication in the final submit.