hpcaitech / ColossalAI-Examples

Examples of training models with hybrid parallelism using ColossalAI
Apache License 2.0
333 stars 102 forks source link

[bert] update the zero pretraining and finetuning with new zero #165

Closed FrankLeeeee closed 1 year ago

FrankLeeeee commented 1 year ago

Fixed the training script such that len(dataloader) works fine. This script is updated with the new zero api as well.