yownas / shift-attention

In stable diffusion, generate a sequence of images shifting attention in the prompt.
Other
164 stars 17 forks source link

Doesn't seem to be changing weights? #8

Closed Mic-N closed 1 year ago

Mic-N commented 1 year ago

Have been experimenting trying to get this to work, following even the example (cat-dog) shift, the values do not appear to be incrementing.. Prompt in SD reading "photo of (cat:1~0) or (dog:0~1)", the prompt extracted from the first png reads: "photo of (cat:1.0) or (dog:0.0)", the final image in the series also shows the same "photo of (cat:1.0) or (dog:0.0)" with the prompt values unchanged, rather than having progressed to cat:0.0 dog:1.0

Interestingly, the image content does seem to be shifting as if the seed is varying, though the seed coming from PNG info shows the same.

Version: b07989d4 (Thu Feb 16 10:08:45 2023) Sample output attached. shift-00006.zip

yownas commented 1 year ago

I checked the file you sent and it looks like it changed the values as it should.

I made this script mostly as a tool to help me poke around and learn more about how prompts work. One thing I've noticed is that "(cat:0)" doesn't necessarily mean "no cat" but rather "there IS a cat, just try to not think about it.". Sometimes you get lucky with the seed and/or model but sometimes you need to help it a little. I've had some luck with adding things to the negative prompt like "kitten" would make "(cat:0)" go away.

But I'm not an expert, just a guy who likes to poke things. :)

(If you update your script you'll get the changes I made yesterday. Then you can write a prompt, generate a handful of pictures (6-7, since this is stable diffusion), set "SSIM threshold" to something like 0.8 and let the script try to fill in the animation to make it smooth by itself.)

Mic-N commented 1 year ago

I could have sworn the values weren't changing.... But yep, you're right.

🤷

I'll pull the new version and get back to playing with it. Nice concept though, I'll keep messing about to see what I can come up with.