search

anarchy-ai / LLM-VM

irresponsible innovation. Try now at https://chat.dev/
https://anarchy.ai/
MIT License
477 stars 147 forks source link

Implement FSDP for training large datasets #218

Open TheRealVish opened 1 year ago

TheRealVish commented 1 year ago

Definition of done: Implement training large models using FSDP to accelerate training on large datasets.

Reference: https://pytorch.org/blog/introducing-pytorch-fully-sharded-data-parallel-api/

lucylililiwang commented 8 months ago

Hi, Can I please work on this issue? Thank you!