search

LeonHLJ / FouriScale

Official implementation of FouriScale (ECCV2024)
Apache License 2.0
128 stars 5 forks source link

Problem about introducing ControlNet to you model #4

Closed Norman-Ou closed 4 months ago

Norman-Ou commented 5 months ago

Hi,

I introduce a ControlNet Module to your model, but the width and height of the latent that be the input to ControlNet are the same of target image width and height.

It is conflict to your setting that latent height(or width)= target height(or width) // 8

LeonHLJ commented 5 months ago

Hi,

I introduce a ControlNet Module to your model, but the width and height of the latent that be the input to ControlNet are the same of target image width and height.

It is conflict to your setting that latent height(or width)= target height(or width) // 8

Thanks for your attention. I can't catch what you said about "the width and height of the latent that be the input to ControlNet are the same as the target image width and height." In Sec. 3.2 of the paper on ControlNet, the height and width of the image condition for ControlNet should be 8× that of the latent input for ControlNet, in order to match the convolution size. If you try to use ControlNet with FouriScale, you should set the width/height of the image condition to be the same as the target image, the width/height of the latent input for ControlNet should also be the same as the latent for UNet.