[Bug]:

[hugo4711]

Is there an existing issue for this?

• [X] I have searched the existing issues and checked the recent builds/commits of both this extension and the webui

What happened?

When using ip_adapter_face_id or ip_adapter_face_id_plus with ip-adapter-plus-face_sdxl_vit_h I get error messages as posted below

2024-06-07 13:14:38,508 - roop - INFO - Restore face with CodeFormer
Total progress: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 24/24 [00:33<00:00,  1.39s/it]
2024-06-07 13:15:13,122 - ControlNet - INFO - unit_separate = False, style_align = False██████████████████████████████████████████████████| 24/24 [00:33<00:00,  1.15s/it]
2024-06-07 13:15:13,122 - ControlNet - INFO - Loading model from cache: ip-adapter-plus-face_sdxl_vit-h [368cf551]
2024-06-07 13:15:13,125 - ControlNet - INFO - Using preprocessor: ip-adapter_face_id
2024-06-07 13:15:13,125 - ControlNet - INFO - preprocessor resolution = 1024
*** Error running process: D:\Program Files\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py
    Traceback (most recent call last):
      File "D:\Program Files\stable-diffusion-webui\modules\scripts.py", line 825, in process
        script.process(p, *script_args)
      File "D:\Program Files\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 1222, in process
        self.controlnet_hack(p)
      File "D:\Program Files\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 1207, in controlnet_hack
        self.controlnet_main_entry(p)
      File "D:\Program Files\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 1162, in controlnet_main_entry
        param.control_model.hook(
      File "D:\Program Files\stable-diffusion-webui\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
        return func(*args, **kwargs)
      File "D:\Program Files\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\ipadapter\plugable_ipadapter.py", line 148, in hook
        *[self.ipadapter.get_image_emb(o) for o in preprocessor_outputs]
      File "D:\Program Files\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\ipadapter\plugable_ipadapter.py", line 148, in <listcomp>
        *[self.ipadapter.get_image_emb(o) for o in preprocessor_outputs]
      File "D:\Program Files\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\ipadapter\ipadapter_model.py", line 347, in get_image_emb
        return self._get_image_embeds(preprocessor_output)
      File "D:\Program Files\stable-diffusion-webui\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
        return func(*args, **kwargs)
      File "D:\Program Files\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\ipadapter\ipadapter_model.py", line 170, in _get_image_embeds
        clip_vision_output["hidden_states"][-2].to(
    IndexError: too many indices for tensor of dimension 2

---

2024-06-07 13:15:57,250 - roop - INFO - Restore face with CodeFormer
Total progress: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 24/24 [00:41<00:00,  1.73s/it]
2024-06-07 13:17:12,712 - ControlNet - INFO - unit_separate = False, style_align = False██████████████████████████████████████████████████| 24/24 [00:41<00:00,  1.24s/it]
2024-06-07 13:17:12,713 - ControlNet - INFO - Loading model from cache: ip-adapter-plus-face_sdxl_vit-h [368cf551]
2024-06-07 13:17:12,715 - ControlNet - INFO - Using preprocessor: ip-adapter_face_id_plus
2024-06-07 13:17:12,715 - ControlNet - INFO - preprocessor resolution = 512
*** Error running process: D:\Program Files\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py
    Traceback (most recent call last):
      File "D:\Program Files\stable-diffusion-webui\modules\scripts.py", line 825, in process
        script.process(p, *script_args)
      File "D:\Program Files\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 1222, in process
        self.controlnet_hack(p)
      File "D:\Program Files\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 1207, in controlnet_hack
        self.controlnet_main_entry(p)
      File "D:\Program Files\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 1162, in controlnet_main_entry
        param.control_model.hook(
      File "D:\Program Files\stable-diffusion-webui\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
        return func(*args, **kwargs)
      File "D:\Program Files\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\ipadapter\plugable_ipadapter.py", line 148, in hook
        *[self.ipadapter.get_image_emb(o) for o in preprocessor_outputs]
      File "D:\Program Files\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\ipadapter\plugable_ipadapter.py", line 148, in <listcomp>
        *[self.ipadapter.get_image_emb(o) for o in preprocessor_outputs]
      File "D:\Program Files\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\ipadapter\ipadapter_model.py", line 347, in get_image_emb
        return self._get_image_embeds(preprocessor_output)
      File "D:\Program Files\stable-diffusion-webui\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
        return func(*args, **kwargs)
      File "D:\Program Files\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\ipadapter\ipadapter_model.py", line 170, in _get_image_embeds
        clip_vision_output["hidden_states"][-2].to(
    TypeError: 'FaceIdPlusInput' object is not subscriptable

---

Steps to reproduce the problem

1. Go to Controlnet > Select Preprocessor and Model
2. Upload an face image
3. Generate
4. Console Log shows the messages above

What should have happened?

Face should be generated / replaced in the generated image

Commit where the problem happens

webui: c91dbe5c3b9377854239310311715b060de82ff1 controlnet:

What browsers do you use to access the UI ?

Google Chrome

Command Line Arguments

--xformers

List of enabled extensions

Console logs

See above

Additional information

No response

[huchenlei]

I don't think we have "ip-adapter-plus-face_sdxl_vit_h" preprocessor. It seems like you are using wrong preprocessor for your ipadapter model. Just select "ipadapter-auto" preprocessor and it will automatically match the correct preprocessor for you.

[hugo4711]

Ok, I downloaded it from https://huggingface.co/h94/IP-Adapter/blob/main/sdxl_models/ip-adapter-plus-face_sdxl_vit-h.safetensors

See: https://github.com/tencent-ailab/IP-Adapter/issues/59

Is that not compatible with the sd-webui-controlnet extension in automatic 1111?