search

Mikubill / sd-webui-controlnet

WebUI extension for ControlNet
GNU General Public License v3.0
16.88k stars 1.94k forks source link

[Bug]: Exception: Insightface: No face found in image. (1.5) #2747

Open ZeroCool22 opened 5 months ago

ZeroCool22 commented 5 months ago

Is there an existing issue for this?

What happened?

When try to use it for Inpainting in img2img i get the follow error (it didn't happen before the last update):

42%|██████████████████████████████████▉                                                | 8/19 [00:03<00:04,  2.23it/s]
2024-04-14 22:08:57,930 - ControlNet - INFO - unit_separate = False, style_align = False
2024-04-14 22:08:58,199 - ControlNet - INFO - Loading model: control_v11p_sd15_inpaint [ebff9138]
2024-04-14 22:09:01,117 - ControlNet - INFO - Loaded state_dict from [C:\Users\ZeroCool22\Desktop\AutoSDXL\stable-diffusion-webui\extensions\sd-webui-controlnet\models\1.5\control_v11p_sd15_inpaint.pth]
2024-04-14 22:09:01,117 - ControlNet - INFO - controlnet_default_config
2024-04-14 22:09:02,808 - ControlNet - INFO - ControlNet model control_v11p_sd15_inpaint [ebff9138](ControlModelType.ControlNet) loaded.
2024-04-14 22:09:02,844 - ControlNet - INFO - Using preprocessor: inpaint
2024-04-14 22:09:02,844 - ControlNet - INFO - preprocessor resolution = 600
2024-04-14 22:09:03,071 - ControlNet - INFO - Loading model: ip-adapter-faceid-plusv2_sd15 [6e14fc1a]
2024-04-14 22:09:03,393 - ControlNet - INFO - Loaded state_dict from [C:\Users\ZeroCool22\Desktop\AutoSDXL\stable-diffusion-webui\models\ControlNet\ip-adapter-faceid-plusv2_sd15.bin]
2024-04-14 22:09:03,727 - ControlNet - INFO - ControlNet model ip-adapter-faceid-plusv2_sd15 [6e14fc1a](ControlModelType.IPAdapter) loaded.
2024-04-14 22:09:03,762 - ControlNet - INFO - Using preprocessor: ip-adapter_face_id_plus
2024-04-14 22:09:03,762 - ControlNet - INFO - preprocessor resolution = 600
*** Error running process: C:\Users\ZeroCool22\Desktop\AutoSDXL\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py
    Traceback (most recent call last):
      File "C:\Users\ZeroCool22\Desktop\AutoSDXL\stable-diffusion-webui\modules\scripts.py", line 825, in process
        script.process(p, *script_args)
      File "C:\Users\ZeroCool22\Desktop\AutoSDXL\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 1245, in process
        self.controlnet_hack(p)
      File "C:\Users\ZeroCool22\Desktop\AutoSDXL\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 1230, in controlnet_hack
        self.controlnet_main_entry(p)
      File "C:\Users\ZeroCool22\Desktop\AutoSDXL\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 995, in controlnet_main_entry
        controls, hr_controls = list(zip(*[preprocess_input_image(img) for img in optional_tqdm(input_images)]))
      File "C:\Users\ZeroCool22\Desktop\AutoSDXL\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 995, in <listcomp>
        controls, hr_controls = list(zip(*[preprocess_input_image(img) for img in optional_tqdm(input_images)]))
      File "C:\Users\ZeroCool22\Desktop\AutoSDXL\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 952, in preprocess_input_image
        detected_map, is_image = self.preprocessor[unit.module](
      File "C:\Users\ZeroCool22\Desktop\AutoSDXL\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\utils.py", line 81, in decorated_func
        return cached_func(*args, **kwargs)
      File "C:\Users\ZeroCool22\Desktop\AutoSDXL\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\utils.py", line 65, in cached_func
        return func(*args, **kwargs)
      File "C:\Users\ZeroCool22\Desktop\AutoSDXL\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\global_state.py", line 37, in unified_preprocessor
        return preprocessor_modules[preprocessor_name](*args, **kwargs)
      File "C:\Users\ZeroCool22\Desktop\AutoSDXL\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\processor.py", line 850, in face_id_plus
        face_embed, _ = g_insight_face_model.run_model(img)
      File "C:\Users\ZeroCool22\Desktop\AutoSDXL\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\processor.py", line 781, in run_model
        face = InsightFaceModel.pick_largest_face(faces)
      File "C:\Users\ZeroCool22\Desktop\AutoSDXL\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\processor.py", line 738, in pick_largest_face
        raise Exception("Insightface: No face found in image.")
    Exception: Insightface: No face found in image.

Steps to reproduce the problem

  1. Go to ....
  2. Press ....
  3. ...

What should have happened?

Detect the face.

Commit where the problem happens

webui: AUTOMATIC1111 version: v1.9.0  •  python: 3.10.6  •  torch: 2.1.2+cu121  •  xformers: 0.0.23.post1  •  gradio: 3.41.2

controlnet: Screenshot_7

What browsers do you use to access the UI ?

Mozilla Firefox

Command Line Arguments

set COMMANDLINE_ARGS=--xformers --api --disable-nan-check --medvram-sdxl

List of enabled extensions

Screenshot_9

Console logs

42%|██████████████████████████████████▉                                                | 8/19 [00:03<00:04,  2.23it/s]
2024-04-14 22:08:57,930 - ControlNet - INFO - unit_separate = False, style_align = False
2024-04-14 22:08:58,199 - ControlNet - INFO - Loading model: control_v11p_sd15_inpaint [ebff9138]
2024-04-14 22:09:01,117 - ControlNet - INFO - Loaded state_dict from [C:\Users\ZeroCool22\Desktop\AutoSDXL\stable-diffusion-webui\extensions\sd-webui-controlnet\models\1.5\control_v11p_sd15_inpaint.pth]
2024-04-14 22:09:01,117 - ControlNet - INFO - controlnet_default_config
2024-04-14 22:09:02,808 - ControlNet - INFO - ControlNet model control_v11p_sd15_inpaint [ebff9138](ControlModelType.ControlNet) loaded.
2024-04-14 22:09:02,844 - ControlNet - INFO - Using preprocessor: inpaint
2024-04-14 22:09:02,844 - ControlNet - INFO - preprocessor resolution = 600
2024-04-14 22:09:03,071 - ControlNet - INFO - Loading model: ip-adapter-faceid-plusv2_sd15 [6e14fc1a]
2024-04-14 22:09:03,393 - ControlNet - INFO - Loaded state_dict from [C:\Users\ZeroCool22\Desktop\AutoSDXL\stable-diffusion-webui\models\ControlNet\ip-adapter-faceid-plusv2_sd15.bin]
2024-04-14 22:09:03,727 - ControlNet - INFO - ControlNet model ip-adapter-faceid-plusv2_sd15 [6e14fc1a](ControlModelType.IPAdapter) loaded.
2024-04-14 22:09:03,762 - ControlNet - INFO - Using preprocessor: ip-adapter_face_id_plus
2024-04-14 22:09:03,762 - ControlNet - INFO - preprocessor resolution = 600
*** Error running process: C:\Users\ZeroCool22\Desktop\AutoSDXL\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py
    Traceback (most recent call last):
      File "C:\Users\ZeroCool22\Desktop\AutoSDXL\stable-diffusion-webui\modules\scripts.py", line 825, in process
        script.process(p, *script_args)
      File "C:\Users\ZeroCool22\Desktop\AutoSDXL\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 1245, in process
        self.controlnet_hack(p)
      File "C:\Users\ZeroCool22\Desktop\AutoSDXL\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 1230, in controlnet_hack
        self.controlnet_main_entry(p)
      File "C:\Users\ZeroCool22\Desktop\AutoSDXL\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 995, in controlnet_main_entry
        controls, hr_controls = list(zip(*[preprocess_input_image(img) for img in optional_tqdm(input_images)]))
      File "C:\Users\ZeroCool22\Desktop\AutoSDXL\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 995, in <listcomp>
        controls, hr_controls = list(zip(*[preprocess_input_image(img) for img in optional_tqdm(input_images)]))
      File "C:\Users\ZeroCool22\Desktop\AutoSDXL\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 952, in preprocess_input_image
        detected_map, is_image = self.preprocessor[unit.module](
      File "C:\Users\ZeroCool22\Desktop\AutoSDXL\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\utils.py", line 81, in decorated_func
        return cached_func(*args, **kwargs)
      File "C:\Users\ZeroCool22\Desktop\AutoSDXL\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\utils.py", line 65, in cached_func
        return func(*args, **kwargs)
      File "C:\Users\ZeroCool22\Desktop\AutoSDXL\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\global_state.py", line 37, in unified_preprocessor
        return preprocessor_modules[preprocessor_name](*args, **kwargs)
      File "C:\Users\ZeroCool22\Desktop\AutoSDXL\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\processor.py", line 850, in face_id_plus
        face_embed, _ = g_insight_face_model.run_model(img)
      File "C:\Users\ZeroCool22\Desktop\AutoSDXL\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\processor.py", line 781, in run_model
        face = InsightFaceModel.pick_largest_face(faces)
      File "C:\Users\ZeroCool22\Desktop\AutoSDXL\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\processor.py", line 738, in pick_largest_face
        raise Exception("Insightface: No face found in image.")
    Exception: Insightface: No face found in image.


### Additional information

_No response_
ZeroCool22 commented 5 months ago

Image used in controlnet:

45889999

PD: I already tried the suggested solutions in other posts, still the same error.

huchenlei commented 5 months ago

Suspected culprit: #2744.

huchenlei commented 5 months ago

I cannot reproduce your issue. InstantID does correctly detects face in the image. image

ZeroCool22 commented 5 months ago

I cannot reproduce your issue. InstantID does correctly detects face in the image. image

I use 1.5, NOT SDXL and with IP-Adapter, not Instant_ID

Screenshot_8

huchenlei commented 5 months ago

SD15 and SDXL uses same preprocessor. IPA and InstantID both uses insightface under the hood. IPA also works fine image

ZeroCool22 commented 5 months ago

Suspected culprit: #2744.

Can you tell me where i add that code line exatcly?

Screenshot_10

huchenlei commented 5 months ago

You can find file change here: https://github.com/Mikubill/sd-webui-controlnet/pull/2744/files

You just need to change 2 lines of code to revert that PR.

huchenlei commented 5 months ago

I think the PR is justified as main instant id repo also has the RGB to BGR convertion before passing the image to insightface here: https://github.com/InstantID/InstantID/blob/042a57c75a265db302eea4e402265d7a803dc93d/gradio_demo/app.py#L233

ZeroCool22 commented 5 months ago

Thx, and this one?:

face_image_cv2 = convert_from_image_to_cv2(face_image)

ZeroCool22 commented 5 months ago

You can find file change here: https://github.com/Mikubill/sd-webui-controlnet/pull/2744/files

You just need to change 2 lines of code to revert that PR.

I did it, but still get same error.

Screenshot_11

It's because i didn't add the other line yet?

(face_image_cv2 = convert_from_image_to_cv2(face_image))

huchenlei commented 5 months ago

Hmmmm, maybe checkout to https://github.com/Mikubill/sd-webui-controlnet/commit/eb451a007f7040288e865f96e9ee0842aa6ef91c and see if the issue is still there?

ZeroCool22 commented 5 months ago

Hmmmm, maybe checkout to eb451a0 and see if the issue is still there?

Did all the changes, but same error.

Anyway, thx to take the time and try to help me.

Instant ID only works for SDXL right?

PD: If you want to see for yourself, we can use AnyDesk if you want sometime.

pluviamtonitrus commented 4 months ago

@huchenlei is there any way to skip or ignore an image if there's no face in it instead of throwing an error?