the output of controlnet tile is awasome, and i want to train a tile for custom dataset. However, the resultis strange.
my training setup:
use the train_controlnet.py script from diffusers.
for the high-resolution image with shape (width, height), we downsample hr img to (width//k, height//k), where k is random sample from {2, 4, 8}, then upsample to (width, height) as controlnet input. and hr img as sd input. I train cn with lr 1e-5 for 20k update steps.
however, the result seems bad, especially in the x4 x8 case, the output appears as a grid. Can you point out where my training Settings are wrong?
the output of controlnet tile is awasome, and i want to train a tile for custom dataset. However, the resultis strange. my training setup: use the train_controlnet.py script from diffusers.
for the high-resolution image with shape (width, height), we downsample hr img to (width//k, height//k), where k is random sample from {2, 4, 8}, then upsample to (width, height) as controlnet input. and hr img as sd input. I train cn with lr 1e-5 for 20k update steps.
however, the result seems bad, especially in the x4 x8 case, the output appears as a grid. Can you point out where my training Settings are wrong?
x2 upsample
x4 upsample
x8 upsample