foundation-model-stack / fms-fsdp

🚀 Efficiently (pre)training foundation models with native PyTorch features, including FSDP for training and SDPA implementation of Flash attention v2.
https://pytorch.org/docs/stable/fsdp.html
Apache License 2.0
116 stars 18 forks source link

Add doc string to the apis in the repo #21

Closed lchu-ibm closed 4 months ago

lchu-ibm commented 4 months ago

We can do only the major ones, it doesn't have to be 100% coverage at this stage. Focus on the ones need doc.