p1atdev / LECO

Low-rank adaptation for Erasing COncepts from diffusion models.
https://arxiv.org/abs/2303.07345
Apache License 2.0
307 stars 23 forks source link

SDXL: tuple index out of range #39

Closed Chadius closed 4 months ago

Chadius commented 5 months ago

Hi, I'm trying to use Google Colab to generate a slider for SDXL. I'm pretty sure my config is wrong but I don't know what the error I see means.

I'm using the cat ears example as a base to make a config file. When I run the script on Google Colab with a A100, it says there are no pooled_embeds. I don't know what that means.

target='1girl' positive='1girl, cat ears' unconditional='' neutral='' action='enhance' guidance_scale=1.0 resolution=1024 dynamic_resolution=False batch_size=2 dynamic_crops=False
Traceback (most recent call last):
  File "/content/LECO/train_lora_xl.py", line 407, in <module>
    main(args)
  File "/content/LECO/train_lora_xl.py", line 394, in main
    train(config, prompts)
  File "/content/LECO/train_lora_xl.py", line 131, in train
    cache[prompt] = PromptEmbedsXL(
  File "/content/LECO/prompt_util.py", line 23, in __init__
    self.pooled_embeds = args[1]
IndexError: tuple index out of range```

Here is my config file, `cat_ears_xl_config.yaml`

```yaml
prompts_file: "./cat_ears_xl_prompts.yaml"

pretrained_model:
  name_or_path: "stabilityai/stable-diffusion-xl-base-1.0"
  v2: true
  v_pred: true

network:
  type: "lierla"
  rank: 16
  alpha: 1.0

train:
  precision: "bfloat16"
  noise_scheduler: "ddim"
  iterations: 2000
  batch_size: 2
  lr: 1e-4
  optimizer: "AdamW"
  lr_scheduler: "constant"

save:
  name: "cat_ears"
  path: "./output/cat_ears_xl"
  per_steps: 200
  precision: "bfloat16"

logging:
  use_wandb: false
  verbose: true

other:
  use_xformers: true

Here is my prompt file, cat_ears_xl_prompts.yaml

- target: "1girl"
  positive: "1girl, cat ears"
  unconditional: ""
  neutral: ""
  action: "enhance"
  guidance_scale: 1.0
  resolution: 1024
  batch_size: 2

And then I run this command: !python LECO/train_lora_xl.py --config_file "./cat_ears_xl_config.yaml"

What am I missing from the config? Is there a setting that is required for SDXL but not 1.5?

ChrisWang13 commented 5 months ago

pretrained_model: name_or_path: "stabilityai/stable-diffusion-xl-base-1.0" v2: False

v2 should be set to False instead of true.

By default, the v2 is set to False. If you specify it to be true, It will alter the model layers.

Chadius commented 4 months ago

Thank you for the advice.

Unfortunately the jax library deprecated some features this project relies on, so I can't build this anymore. I'll close this issue and open a new one to address it.