LibAI_bert_large_pretrain_graph_nl24_nah16_hs1024_FP16_actrue_DP16_MP1_PP1_zerotrue_stage2_mbs32_gbs512_acc1_2n8g

[Tendo33]

NVIDIA_GeForce_RTX_3080_Ti | master@021e5e62 | rank_per_process @021e5e62 | naive@021e5e62 -- | -- | -- | -- LibAI_bert_large_pretrain graph_nl24_nah16_hs1024_FP16_actrue DP16_MP1_PP1_zerotrue_stage2 _mbs32_gbs512_acc1_2n8g | building plan Done! Cost time: 23.85s. building graph Done! Cost time: 11.94s node0:4536MIB–4536MIB node1:3590MIB–3590MIB | building plan Done! Cost time: 24.18s. building graph Done! Cost time: 12.23 s. node0: 8814MIB--8960MIB node1:3638MIB–3638MIB | building plan Done! Cost time: 24.34s. building graph Done! Cost time: 12.21s. node0:5774MIB--5864MIB node1:4242MIB–4262MIB

• 全局loss曲线对比

• 50步loss曲线对比

• 100步loss曲线对比