search

AUTOMATIC1111 / stable-diffusion-webui

Stable Diffusion web UI
GNU Affero General Public License v3.0
142.53k stars 26.88k forks source link

Change of models in the middle of the image generation #1901

Closed morganavr closed 1 year ago

morganavr commented 2 years ago

Waifu Diffusion and NovelAI (private) models know many danbooru tags, such as "leg_up", "knees_together_feet_apart", "kneeling" not recognized by standard SD model. Prompt editing already supports feature of changing part of prompts on the run with syntax [woman:man:0.2]. Would not it be possible to change models (ckpt files) as well?

I think the prompt syntax can be like this: [waifu.ckpt:"1girl, leg up, masterpiece, bikini";default_sd.ckpt:"photo of young woman wearing bikini, kodak portra 400";0.6]

It will work this way - first "waifu.ckpt" will be used to generate 60% of steps, then model will be changed to "default_sd.ckpt" and remaining steps will be processed with it. I hope that this change of models in runtime will result in correct pose (not possible to achieve with default SD) and then style will be changed from anime to realistic in the remaining 40% steps.

bmaltais commented 2 years ago

Could you not achieve this manually by switching the model and doing the 2nd part in img2img?

utility-ovo commented 2 years ago

i exchange Waifu Diffusion to NovelAI model，and try to use img2img .But the cmd only told me“Draw mask' is not in list”.How to
solve this problem？

catboxanon commented 1 year ago

Would be resolved by https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/12371, https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/12377