Tendo33 / oneflow-test

oneflow test
0 stars 0 forks source link

火山引擎 libai_gpt 与 megatron_gpt 对比测试 #13

Open Tendo33 opened 1 year ago

Tendo33 commented 1 year ago
NVIDIA A100-SXM4-80GB OneFlow_0d4bc37 Megatron_e156d2f
gpt2_pretrain_graph_nl60_nah144_hs2304_FP16actrue
DP16_MP1_PP1_zerofalse_stage2_mbs2_gbs32_acc1_2n8g
80290-80370 MiB /29.15 samples/s 77944 - 78072 MiB /26.4 samples/s
gpt2_pretrain_graph_nl60_nah144_hs2304_FP16actrue
DP16_MP1_PP1_zerofalse_stage2_mbs2_gbs32_acc4_2n8g
78316-78573 MiB/33 samples/s 77944 - 78072 MiB /33.8 samples/s