search

foundation-model-stack / fms-fsdp

🚀 Efficiently (pre)training foundation models with native PyTorch features, including FSDP for training and SDPA implementation of Flash attention v2.
https://pytorch.org/docs/stable/fsdp.html
Apache License 2.0
162 stars 27 forks source link

Enhance wandb support #46

Closed lchu-ibm closed 6 months ago

lchu-ibm commented 6 months ago
  1. expose wandb_project_name in configurations.
  2. add memory tracking in wandb logging.