search

ljleb / sd-webui-freeu

a1111 implementation of https://github.com/ChenyangSi/FreeU
MIT License
301 stars 16 forks source link

Question about interaction with Stable Diffusion Dynamic Thresholding (CFG Scale Fix) #40

Closed FBarrionuevoIT closed 9 months ago

FBarrionuevoIT commented 10 months ago

Is it possible that this extension conflicts with Stable Diffusion Dynamic Thresholding (CFG Scale Fix)? I have noticed that this extension causes that in certain occasions with high CFG or when used with some LoRa the colors look very saturated with high contrast.

The extension is from this repository: https://github.com/mcmonkeyprojects/sd-dynamic-thresholding

for example: FreeU: Off CFG Scale Fix: On (Value:4) image

FreeU: On CFG Scale Fix: On (Value:4) image

ljleb commented 10 months ago

The different FreeU presets have been tested against the vanilla pipeline AFAIK. My understanding here is, you probably want to tweak the parameters of FreeU when used in conjunction with other extensions.

Arguably, IMO, some aspects of the images with freeu enabled are more aesthetic than without in the batch you shared.

FBarrionuevoIT commented 10 months ago

I see... I realized about this when instead of creating an image with noticeable contrast and vivid colors I wanted to create images with a lower, more natural colors and contrasts, similar to a not so... old color photo. I see that the way the character looks when FreeU is activated is clearer, more detailed and beautiful, but I feel that control is lost in these other aspects in this case. In the images this is quite noticeable in the top left image and the bottom left image, where instead of looking like a normal color photo it looks more like a cinematic render.

So I thought it was a good idea to come and ask to see if there was something I was doing wrong or if the extension that allows to correct the contrast and color of the generated images to lower, balanced and less saturated values imitating what happens when you change the cfg value had some conflict with this extension.

I love this extension and would like to be able to use it properly together with ADetailer and CFG Scale Fix, which are the extensions I love to use the most together with controlNet. I'm excited to see what things can be added to FreeU over time.

I thank you for the answer and I remain attentive for advice and suggestions.

ljleb commented 10 months ago

Well again, IMO, you should play with the settings if you aren't satisfied with the results. FreeU isn't only an "enabled" button ;)

You can save presets for this purpose.

For example, you may try to reduce the backbone scale or bring skip scale closer to 1 and see what happens. Maybe instead try stopping early or starting later in the schedule using the sliders at the top of the extension ui. I don't have an intuition to predict what each will visually look like unfortunately.

I'm interested if anyone else has other ideas for this.

6DammK9 commented 10 months ago

Just tweak all the numbers again. I have tested the interaction between FreeU along with Dynamic CFG, and it can have big difference to make all previous optimal numbers obselete. Unfourtunately, the new optimal value would be out of expactation. I don't think there is a bug, any optimal numercal value (hyperparamaters) in this field only works for a specific use case. https://github.com/6DammK9/nai-anime-pure-negative-prompt/blob/main/ch01/freeu.md

ljleb commented 10 months ago

As a side note, you may want to know @6DammK9 that the recommended values for V2 are not the same as the recommended values for V1. The author has not finalized the recommendations, I am waiting for them before updating the code.