search

pytorch / xla

Enabling PyTorch on XLA Devices (e.g. Google TPU)
https://pytorch.org/xla
Other
2.5k stars 483 forks source link

Improve FSDP doc #4281

Open JackCaoG opened 1 year ago

JackCaoG commented 1 year ago

📚 Documentation

There were some common questions in FSPD regarding how to wrap the model, how flatten_parameters works etc. I think we should add a FAQ section to the https://github.com/pytorch/xla/blob/master/docs/fsdp.md to address those questions.

@Liyang90 @AlexWertheim @ronghanghu

alanwaketan commented 1 year ago

@AlexWertheim LMK if I wrongly assigned this to you.

AlexWertheim commented 1 year ago

@alanwaketan nope, looks good to me. I'll work with @Liyang90 and @ronghanghu to collect information, but I can be ultimately responsible for updating the documentation. Thanks!