Getting AttributeError: 'Linear' object has no attribute 'set_lora_layer' when trying to train the subject dreambooth

[iliasmiraoui]

Trying to run the script in a colab instance and getting the following error which seems odd:

%env MODEL_NAME=stabilityai/stable-diffusion-xl-base-1.0
%env OUTPUT_DIR=lora-sdxl-dog
%env INSTANCE_DIR=dog
%env PROMPT=a sbu dog
%env VALID_PROMPT=a sbu dog in a bucket

!accelerate launch train_dreambooth_lora_sdxl.py --pretrained_model_name_or_path=$MODEL_NAME --instance_data_dir=$INSTANCE_DIR  --output_dir=$OUTPUT_DIR --instance_prompt="${PROMPT}" --rank=64 --resolution=1024 --train_batch_size=1 --learning_rate=5e-5 --report_to="wandb" --lr_scheduler="constant" --lr_warmup_steps=0 --max_train_steps=1000 --validation_prompt="${VALID_PROMPT}" --validation_epochs=50 --seed="0" --mixed_precision="fp16"  --gradient_checkpointing --use_8bit_adam

Traceback (most recent call last): File "/content/ziplora-pytorch/train_dreambooth_lora_sdxl.py", line 2004, in main(args) File "/content/ziplora-pytorch/train_dreambooth_lora_sdxl.py", line 1123, in main attn_module.to_q.set_lora_layer( File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1695, in getattr raise AttributeError(f"'{type(self).name}' object has no attribute '{name}'") AttributeError: 'Linear' object has no attribute 'set_lora_layer'

[iliasmiraoui]

Found the issue and fixed it: https://github.com/huggingface/diffusers/issues/5919

[Jamie-Cheung]

Found the issue and fixed it: huggingface/diffusers#5919

Whether I uninstall peft or not, I am still facing this bug, can someone tell me the solution. Can you tell me the detail of your enviroment？