Examples

[pansershrek]

Hello! Is any examples where classic FSDP replacement with YaFSDP ( where model trained without accelerate or other libraries, but with manual FSDP setup)? Is it even possible to replace FSDP with YaFSDP without changing other libraries or scripts?

[antony-frolov]

@pansershrek hello! There are no more examples of training with YaFSDP as of now, but you can certainly use our patches to accelerate and transformers libraries as a reference. We've tried to make YaFSDP interface similar to FSDP one, so the transition shouldn't be too difficult.

[pansershrek]

Ok, thanks for answer.