instruction for dreambooth is missing

[MightyChaos]

Dear auhtors, I am trying to run image-sculpting on my own images. It seems we need LoRA finetuned models based on the editing script, however in readme, there is no instruction on how to proceed such step. Could you kindly update this part on which code base, and which hyperparameter to set for LoRA? Thank you!

[domejiraphon]

Hi! Yes, we need dreambooth to utilize the input image. We use the implementation from diffusers which can be found here. Basically, clone the repo and train with this command:

export MODEL_NAME="stabilityai/stable-diffusion-xl-base-1.0"
export INSTANCE_DIR="dog"
export OUTPUT_DIR="lora-trained-xl"
export VAE_PATH="madebyollin/sdxl-vae-fp16-fix"

accelerate launch train_dreambooth_lora_sdxl.py \
  --pretrained_model_name_or_path=$MODEL_NAME  \
  --instance_data_dir=$INSTANCE_DIR \
  --pretrained_vae_model_name_or_path=$VAE_PATH \
  --output_dir=$OUTPUT_DIR \
  --mixed_precision="fp16" \
  --instance_prompt="a photo of sks dog" \
  --resolution=1024 \
  --train_batch_size=1 \
  --gradient_accumulation_steps=4 \
  --learning_rate=1e-4 \
  --report_to="wandb" \
  --lr_scheduler="constant" \
  --lr_warmup_steps=0 \
  --max_train_steps=500 \
  --validation_prompt="A photo of sks dog in a bucket" \
  --validation_epochs=25 \
  --seed="0" \
  --push_to_hub`