When trying to use Accelerate with FSDP I get the error:
[rank0]: Traceback (most recent call last):
[rank0]: File "/home/acidhax/dev/SVD_Xtend/train_svd.py", line 1255, in <module>
[rank0]: main()
[rank0]: File "/home/acidhax/dev/SVD_Xtend/train_svd.py", line 881, in main
[rank0]: unet, optimizer, lr_scheduler, train_dataloader = accelerator.prepare(
[rank0]: File "/home/acidhax/miniconda3/envs/training/lib/python3.10/site-packages/accelerate/accelerator.py", line 1292, in prepare
[rank0]: result = tuple(
[rank0]: File "/home/acidhax/miniconda3/envs/training/lib/python3.10/site-packages/accelerate/accelerator.py", line 1293, in <genexpr>
[rank0]: self._prepare_one(obj, first_pass=True, device_placement=d) for obj, d in zip(args, device_placement)
[rank0]: File "/home/acidhax/miniconda3/envs/training/lib/python3.10/site-packages/accelerate/accelerator.py", line 1169, in _prepare_one
[rank0]: return self.prepare_model(obj, device_placement=device_placement)
[rank0]: File "/home/acidhax/miniconda3/envs/training/lib/python3.10/site-packages/accelerate/accelerator.py", line 1443, in prepare_model
[rank0]: self.state.fsdp_plugin.set_auto_wrap_policy(model)
[rank0]: File "/home/acidhax/miniconda3/envs/training/lib/python3.10/site-packages/accelerate/utils/dataclasses.py", line 1182, in set_auto_wrap_policy
[rank0]: raise Exception("Could not find the transformer layer class to wrap in the model.")
[rank0]: Exception: Could not find the transformer layer class to wrap in the model.
When trying to use Accelerate with FSDP I get the error: