Some doubts,
from the training script you released, I found that these parameters(down_blocks.1.attentions.0.transformer_blocks.0.attn2.to_out.0.weight,...) in unet were also trained at the same time, so can the trained controlnet be well applied to the downstream model( like SDXL Yamer's Realistic 5)? or will be training another controlnet to adapt the downstream model?
aidings
commented
3 weeks ago
Some doubts, from the training script you released, I found that these parameters(down_blocks.1.attentions.0.transformer_blocks.0.attn2.to_out.0.weight,...) in unet were also trained at the same time, so can the trained controlnet be well applied to the downstream model( like SDXL Yamer's Realistic 5)? or will be training another controlnet to adapt the downstream model?