Mikubill / sd-webui-controlnet

WebUI extension for ControlNet
GNU General Public License v3.0
16.41k stars 1.9k forks source link

[Bug]: Segmentation generated behavior not as expected #2932

Closed alexknows closed 1 month ago

alexknows commented 1 month ago

Is there an existing issue for this?

What happened?

I recently came across OlivioTutorials video on ControlNet's segmentation, but it seems that the generated images is not adhering to it.

Steps:

  1. I generated the map in the ControlNet's section.
  2. I Downloaded map and made a few small tweaks in Photoshop. (e.g to remove a piece of wall and an object)
  3. I replaced the original with the map.
  4. I added the prompts and adjusted all the settings accordingly.
  5. The image it generates has no similarity what so ever to the map I provided in ControlNet.

It seems as if ControlNet is not even enabled (but it is). No errors are thrown.

Steps to reproduce the problem

  1. I generated the map in the ControlNet's section using an image of a bathroom.
  2. I Downloaded map and made a few small tweaks in Photoshop. (e.g to remove a piece of wall and an object)
  3. I replaced the original with the map.
  4. I added the prompts and adjusted all the settings accordingly.
  5. The image it generates has no similarity what so ever to the map I provided in ControlNet.

First, I added the image to controlnet: herringbone-floor-bathroon-2

I generated the following map: orginal-map

I saved the map and made minor tweaks to it in photoshop: segmentation-ad-bathroom

I added the prompt: RAW photo, a modern bathroom with a tub, sink, toilet, center area with a wooden two-tone flooring tile, a jigsaw puzzle style flooring, isometric view, (high detailed), 8k uhd, high quality

Negative prompt semi-realistic, cgi, 3d, render, sketch, cartoon, drawing, anime, duplicates

Adjusted all the controls:

Screenshot 2024-05-24 at 8 28 46 PM

As you can see in the above screenshot the generated image does not match at all the map in the ControlNet.

What should have happened?

Generated image should match the map.

Commit where the problem happens

webui: controlnet:

What browsers do you use to access the UI ?

Google Chrome

Command Line Arguments

No

List of enabled extensions

Screenshot 2024-05-24 at 8 38 37 PM

Console logs

NA

Additional information

No response

huchenlei commented 1 month ago

See https://github.com/Mikubill/sd-webui-controlnet/discussions/2931