54rt1n / ComfyUI-DareMerge

ComfyUI powertools for SD1.5 and SDXL model merging
GNU General Public License v3.0
67 stars 10 forks source link

How to use the simple mask and mask edit nodes? #15

Open Gyramuur opened 2 months ago

Gyramuur commented 2 months ago

It seems that masking allows you to use a model as a "mask". However, what I do not understand is what that actually means or what it functionally does.

I notice in the mask edit node that there is a field to type in layers. Does this allow you to take tags from one model and effectively put them into the other? How does that work?

Do I plug in the model I want to use into the simple masker, then that into mask edit, then type in the tags I want to keep separated by commas to the layers field?

What do I do with the other parameters after the fact? Like what do "arg_one" and "arg_two" do? Also, do I set operation to "true" if I want to merge in those tags?

Or am I misunderstanding how these nodes work?

54rt1n commented 2 months ago

The purpose of the masker is to generate some parameter selection criteria to inform what to merge.

The documentation for the arg_one and arg_two is buried in the code: https://github.com/54rt1n/ComfyUI-DareMerge/blob/master/components/mask_model.py#L225

I think you might have a bit of a misunderstanding on what is going on, so you have to consider what DARE-TIES is doing, and why this process works. It is two concepts, DARE and TIES put together. It's more like DARE w/ TIES.

DARE is a stochastic process which randomly picks some percentage of the parameters, and then does a spherical linear interpolation. TIES says only allow parameters to be selected where the sign of the parameters is the same. The idea being, since both models were trained on the same base model, and there is a ridiculous amount of redundancy in the learned parameters, that you can take a random selection from one and put it in to the other to gain some of the features.

The mask edit simply allows you more fine grained control over what layers to select for this operation.

miasik commented 1 month ago

The documentation for the arg_one and arg_two is buried in the code:

I've tried to understand how "Magnitude Masker" work from code and I understood that it isn't clear for me. If I set a threshold as ".75 + below" does it mean that all below .75 is protected or allowed to merge? Please, add more details to your nodes descriptions.