search

lllyasviel / ControlNet

Let us control diffusion models!
Apache License 2.0
30.52k stars 2.74k forks source link

Is ControlNet suitable for style transferring? #710

Closed CuteZombie7 closed 2 months ago

CuteZombie7 commented 2 months ago

I'd like to generate pictures with a certain style (e.g. Van Gogh's painting style or real-life style). Is ControlNet suitable to do that? For example, I input Van Gogh's "The Starry Night" to ControlNet, and input "dog" and something else as prompts, will ControlNet output a reasonable dog picture just like Van Gogh's painting? I notice that all controls that ControlNet applies to SD model are concrete concepts, such as pic's edge or human's pose. Will ControlNet work well on abstract concepts (like painting style)? Or is there any good tools for extracting pic's style? And then maybe I can train a ControlNet model with extracted styles. I don't want an answer like: Just do it reversely, input a dog picture and "Van Gogh" as prompt. Since there are many styles that cannot be described by text or cannot be input as a prompt.

CuteZombie7 commented 2 months ago

Never mind, just found T2I-Adapter on huggingface. There's no problem now.