[Wandb] Refine wandb logging function

microsoft / Megatron-DeepSpeed

Ongoing research training transformer language models at scale, including: BERT & GPT-2

Other

1.89k stars 344 forks source link

[Wandb] Refine wandb logging function #416

Closed billishyahao closed 3 months ago

billishyahao commented 4 months ago

This patch aims to enable wandb initilization and make it more unified with old tensorboard code style. With this patch enabled, we can log and visilize our training procedure on wandb panel. Here is the example figure

Screenshot 2024-07-09 110418 Screenshot 2024-07-09 110505

billishyahao commented 4 months ago

@microsoft-github-policy-service agree company="Intel"