search

yownas / seed_travel

Small script for AUTOMATIC1111/stable-diffusion-webui to create images between two seeds
Other
307 stars 25 forks source link

Not transitioning well with SD2 model #25

Closed brian000 closed 1 year ago

brian000 commented 1 year ago

It looks like the transition between seeds appears to be broken. I'm not sure if it's SD2 or something with Auto1111. I've tried a run of:

It creates the first 30 images without issue and mostly the same (the variation is visible), but when it comes around to image 31 (when it starts a new seed), it's wildly different. I've uploaded the first 60 output frames up to Mega (since imgur hates webp); you can clearly see where it abruptly cuts to a new seed rather than slowly transitioning over.

For what it's worth, I'm running the latest (as of now) auto1111 at 4b3c5bc. I'm really not sure if it's an auto1111 issue or something with the SD2 model, but the script is definitely not functioning like it used to.

brian000 commented 1 year ago

I was able to figure out how exactly this works: it sets a main seed and a variation seed, and then sets that variation seed strength.
Each frame except for the "pure" ones has a variation seed in it, which is the next "pure" seed it will use. As frames are rendered, the variation seed strength goes up until it reaches the next pure seed.

I've tested another run with SD1.5, and it doesn't seem to be caused by SD2. It did however work just fine with DPM++2M and Euler on both SD1.5 and 2, so I'll blame the DPM++ SDE sampler not playing nice with this script (like Euler a).

yownas commented 1 year ago

Sorry for the late reply. Yes, it seems like some samplers doesn't play nice with variation seeds. I think it might have something to do with this part: https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/4b3c5bc24bffdf429c463a465763b3077fe55eb8/modules/processing.py#L348

That some samplers need random noise (I think), but it isn't treated the same way as the noise you get from the seed(s). Hopefully I'll find time someday to dig further and try to understand what it does and if you can fix it somehow.