Texture Condition

[ArefMYTB]

I want to train a custom condition to transfer texture of a cloth in conditioning image, and do a virtual try-on for a human image input. I tried so many ways. I even tried to transfer color of a simple rectangle. even after 200k steps with 10k data, the results are not good at all. then I tried to align the position of both rectangles by setting conditioning image same as input image. even when I'm using one image for both condition and input, still it cant generate same rectangle.

Plz help me. this is making me crazy.

Thx

[geroldmeisinger]

I can't quiet follow your description, could you please provide some example images of what you want to achieve. it sounds as if you want to use a texture-only (like a picture of a linen-structure?) conditioning image to solely guide the generation of cloths of humans in the inference (like the pose-model but only for texture?). or are you using the control net with img2img?

have you tried to train a simpler control net concept first? you can follow my tutorial at https://civitai.com/articles/2078 and my experiences I had with training a simple edge map control net at https://github.com/lllyasviel/ControlNet/discussions/318#discussioncomment-7092374

related links that could interest you:

• https://github.com/lllyasviel/ControlNet/discussions/412
• https://github.com/lllyasviel/ControlNet/issues/333
• https://github.com/lllyasviel/ControlNet/discussions/440
• https://github.com/lllyasviel/ControlNet/discussions/494
• https://huggingface.co/phucleh2/controlnet_mask2cloth
• Composer: Creative and Controllable Image Synthesis with Composable Conditions

why is everyone so obsessed with fashion try on anyway? :)

[ArefMYTB]

lets say we have a human image below:

I don't care about size, but I want to get color or texture of this cloth and and put it on the woman's cloth.

something like virtual try-on. but for now I just want to transfer color or texture of a region in conditioning image and match it for a region in input image.

[geroldmeisinger]

have you tried to mask the target image in inpainting and use control net color to guide the color?

[geroldmeisinger]

could you please also share the workflow you tried for your control net training, like what did all the conditioning images look like?

[ArefMYTB]

Is there a controlnet color??!! you mean this? I'm not sure if it's work as it keeps location of colors. yeah I've tried inpainting as my base model and as a condition I tried a cloth image, same image as input and also different UNets.

[geroldmeisinger]

https://huggingface.co/thibaud/controlnet-sd21-color-diffusers the location of colors would be constrained by the inpainting mask

jinxixiang/color_controlnet

i didn't know about this, but if you try, please share your experiences

[ArefMYTB]

this is too random as it generated a new person.