search

hpcaitech / ColossalAI-Examples

Examples of training models with hybrid parallelism using ColossalAI
Apache License 2.0
334 stars 102 forks source link

Added hybrid parallel ViLT #29

Closed ExtremeViscent closed 2 years ago

ExtremeViscent commented 2 years ago

An MPI-backended Colossal-AI implementation of Vision-and-Language Transformer using 1D tensor parallel and data parallel.

FrankLeeeee commented 2 years ago

Thanks! Let me try to run this example and take some time to review.

ExtremeViscent commented 2 years ago

Thank you for reviewing. The commit is edited as requested.

binmakeswell commented 2 years ago

Hi @ExtremeViscent Could you cancel the review request for FrankLeeeee? This prevents other reviewers' action.