Support `use_flash_attention_2` under `fsdp_low_cpu_init` option

iwiwi / epochraft-hf-fsdp

Example of using Epochraft to train HuggingFace transformers models with PyTorch FSDP

MIT License

12 stars 5 forks source link

Open iwiwi opened 1 year ago

iwiwi commented 1 year ago