sd-fabric / fabric

MIT License
320 stars 19 forks source link

How is the pos/neg weight set? #10

Open ZichengDuan opened 10 months ago

ZichengDuan commented 10 months ago

Hi, I appreciate your work, and your paper is well written and easy to understand, but there's one issue I haven't found the answer for. I have read through the code and paper but still have no clue about how the weight of positive/negative weight is set. I can see that they depend on the time steps and classifier-free guidance, but still don't know the details. Can you please provide some insights regarding this? Much appreciated.

dvruette commented 10 months ago

Hi, thanks for the kind words! The weight of positive feedback is basically a scaled and shifted square function, with the min_weight usually being set to 0 or a small value like 0.1. Here's an illustration: image

The feedback strength for negative feedback is set as a fraction of the positive weight, usually 0.5. So the shape is the same, it's just the scale that's different.

Looking at the code here, it really is quite convoluted (and also implements some additional features), but the SD webui extension is much cleaner and IMO easier to understand: https://github.com/dvruette/sd-webui-fabric/blob/main/scripts/patching.py#L56

I hope I was able to answer your question, lmk if there's anything that's unclear still!