When H800 is trained with FP8, the performance is not significantly improved compared to FP16, and is even worse than FP16.

[yangzhipeng1108]

Your question Ask a clear and concise question about Megatron-LM.

[LiaoYuanF]

hello，Please tell me, do you know how to start fp8 training? thanks

[yangzhipeng1108]

    --bf16 \
    --fp8-format hybrid \
    --fp8-amax-compute-algo max \
    --fp8-amax-history-len 1024 \
    --transformer-impl transformer_engine

[LiaoYuanF]

    --bf16 \
    --fp8-format hybrid \
    --fp8-amax-compute-algo max \
    --fp8-amax-history-len 1024 \
    --transformer-impl transformer_engine

thanks，I occur the same performance degradation problem as you on H20 GPU, please check your transformer engine version, upgrading to the new version may solve your problem