Regional Seed doesn't seem to work with some model.

ltdrdata / ComfyUI-Inspire-Pack

This repository offers various extension nodes for ComfyUI. Nodes here have different characteristics compared to those in the ComfyUI Impact Pack. The Impact Pack has become too large now...

GNU General Public License v3.0

447 stars 52 forks source link

Regional Seed doesn't seem to work with some model. #59

Closed boonyasukd closed 10 months ago

boonyasukd commented 10 months ago

On my current machine, I'm using Dreamshaper XL Turbo. I downloaded your example png file, and tried out the workflow on my local machine. The change doesn't seem to limited to any particular region, and the change is rather subtle, almost as if there wasn't any noise being injected.

From the workflow, it seems you are using a non-turbo SDXL. It's a bit big to download for my part, so I would like to get a confirmation from you instead that this feature actually doesn't work on every kind of model. Stating that in a README would be fine.

Thank you.

ltdrdata commented 10 months ago

The seed only affects the initial noise configuration. It is unrelated to the model.

boonyasukd commented 10 months ago

I have other models on hand, so I did a bit of experimenting. The result is rather interesting, to say the least... It'd be great if you can guide me what I need to do to proceed.

For the record, I'm using dreamshaper_8.safetensors. I believe this is the SD1.5 version.
The person in the original image is missing a limb, so I made a small masking dot where the limb should be.
Then I added the strength of 0.01.
And I got this result. Injecting additional strength seems to affect the whole image instead of being localized to a particular region:

ltdrdata commented 10 months ago

The regional seed only changes the noise; it is not an inpaint. It changes the local initial state to alter the direction of image generation.

And the masked region is only applied to the place where noise is changed; it is not applied to the diffusion process, so the areas affected by this noise are the entire image. It simply has the most significant impact at the origin.