search

ChenyangSi / FreeU

FreeU: Free Lunch in Diffusion U-Net (CVPR2024 Oral)
https://chenyangsi.top/FreeU/
MIT License
1.59k stars 57 forks source link

SDXL parameters #9

Open nachocheeseburger opened 9 months ago

nachocheeseburger commented 9 months ago

Forgot to include SDXL parameters.

FurkanGozukara commented 9 months ago

I tested SDXL and not good

https://twitter.com/GozukaraFurkan/status/1706253092822376521

ChenyangSi commented 9 months ago

https://wandb.ai/nasirk24/UNET-FreeU-SDXL/reports/FreeU-SDXL-Optimal-Parameters--Vmlldzo1NDg4NTUw?accessToken=6745kr9rjd6e9yjevkr9bpd2lm6dpn6j00428gz5l60jrhl3gj4gubrz4aepupda

FurkanGozukara commented 9 months ago

https://wandb.ai/nasirk24/UNET-FreeU-SDXL/reports/FreeU-SDXL-Optimal-Parameters--Vmlldzo1NDg4NTUw?accessToken=6745kr9rjd6e9yjevkr9bpd2lm6dpn6j00428gz5l60jrhl3gj4gubrz4aepupda

it is missing so many settings exists here

image

NasirKhalid24 commented 9 months ago

FreeU with SDXL finetune and LoRA models needs lower b vales in my experience - finetuning the model encourages it to rely more on backbone

t00350320 commented 2 months ago

with sdxl model 

#model_id = "stabilityai/stable-diffusion-2-1"
model_id = f"RealVisXL_V4.0_Lightning.safetensors"
# model_id = "./stable-diffusion-2-1"
#pip_2_1 = StableDiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float16)
pip_2_1 = StableDiffusionXLPipeline.from_single_file(model_id, torch_dtype=torch.float16)
pip_2_1 = pip_2_1.to("cuda")

got this errors 

 File "/opt/conda/lib/python3.10/site-packages/diffusers/pipelines/stable_diffusion_xl/pipeline_stable_diffusion_xl.py", line 1216, in __call__
    noise_pred = self.unet(
  File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/opt/conda/lib/python3.10/site-packages/diffusers/models/unet_2d_condition.py", line 1188, in forward
    sample = upsample_block(
  File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
TypeError: register_free_upblock2d.<locals>.up_forward.<locals>.forward() got an unexpected keyword argument 'scale'