search

Mikubill / sd-webui-controlnet

WebUI extension for ControlNet
GNU General Public License v3.0
17.04k stars 1.96k forks source link

[Bug]: TypeError: 'FaceIdPlusInput' object is not subscriptable #2730

Closed ElSchulzoML closed 7 months ago

ElSchulzoML commented 7 months ago

Is there an existing issue for this?

What happened?

Hello gentle reader. I hope you can help. I'm trying different combinations of IP-Adapter FaceID without success.

Steps to reproduce the problem

  1. Create Txt2Img prompt.
  2. Choose LCM Sampler and settings. (I also have FreeU and NegPiP enabled)
  3. Activate Controlnet IP-Adapter
  4. Choose these settings:

image

  1. Choose a single image for IP-Adapter
  2. Click Generate

What should have happened?

Generation without TypeError.

Commit where the problem happens

Stable Diffusion Webui DirectML: 25a3b6cbeea8a07afd5e4594afc2f1c79f41ac1a controlnet: eb451a007f7040288e865f96e9ee0842aa6ef91c

What browsers do you use to access the UI ?

Brave

Command Line Arguments

--use-directml --disable-nan-check --use-cpu codeformer --medvram --no-half

List of enabled extensions

image

Console logs

PS D:\Stable Diffusion\stable-diffusion-webui-directml> py launch.py --use-directml --disable-nan-check --use-cpu codeformer --medvram --no-half
fatal: No names found, cannot describe anything.
Python 3.10.11 (tags/v3.10.11:7d4cc5a, Apr  5 2023, 00:38:17) [MSC v.1929 64 bit (AMD64)]
Version: 1.8.0-RC
Commit hash: 25a3b6cbeea8a07afd5e4594afc2f1c79f41ac1a
no module 'xformers'. Processing without...
no module 'xformers'. Processing without...
No module 'xformers'. Proceeding without it.
D:\Stable Diffusion\stable-diffusion-webui-directml\SDenv\lib\site-packages\pytorch_lightning\utilities\distributed.py:258: LightningDeprecationWarning: `pytorch_lightning.utilities.distributed.rank_zero_only` has been deprecated in v1.8.1 and will be removed in v2.0.0. You can import it from `pytorch_lightning.utilities` instead.
  rank_zero_deprecation(
Launching Web UI with arguments: --use-directml --disable-nan-check --use-cpu codeformer --medvram --no-half
ONNX: selected=DmlExecutionProvider, available=['DmlExecutionProvider', 'CPUExecutionProvider']
==============================================================================
You are running torch 2.0.0+cpu.
The program is tested to work with torch 2.1.2.
To reinstall the desired version, run with commandline flag --reinstall-torch.
Beware that this will cause a lot of large files to be downloaded, as well as
there are reports of issues with training tab on the latest version.

Use --skip-version-check commandline argument to disable this check.
==============================================================================
CivitAI Browser+: Aria2 RPC started
fatal: No names found, cannot describe anything.
ControlNet preprocessor location: D:\Stable Diffusion\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\annotator\downloads
2024-04-03 12:30:12,525 - ControlNet - INFO - ControlNet v1.1.443
2024-04-03 12:30:12,751 - ControlNet - INFO - ControlNet v1.1.443
[sd-webui-freeu] Controlnet support: *enabled*
Loading weights [d65bb3a735] from D:\Stable Diffusion\stable-diffusion-webui-directml\models\Stable-diffusion\cyberrealistic_classicV31.safetensors
Creating model from config: D:\Stable Diffusion\stable-diffusion-webui-directml\configs\v1-inference.yaml
2024-04-03 12:30:14,313 - ControlNet - INFO - ControlNet UI callback registered.
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
IIB Database file has been successfully backed up to the backup folder.
Startup time: 10.8s (prepare environment: 13.8s, initialize shared: 2.9s, list SD models: 0.3s, load scripts: 3.2s, create ui: 1.7s, gradio launch: 0.5s).
Loading VAE weights specified in settings: D:\Stable Diffusion\stable-diffusion-webui-directml\models\VAE\vae-ft-mse-840000-ema-pruned.safetensors
Applying attention optimization: sub-quadratic... done.
Model loaded in 22.1s (load weights from disk: 0.4s, create model: 0.9s, apply weights to model: 18.4s, load VAE: 1.7s, load textual inversion embeddings: 0.2s, calculate empty prompt: 0.5s).
2024-04-03 12:31:18,006 - ControlNet - INFO - unit_separate = False, style_align = False
2024-04-03 12:31:18,240 - ControlNet - INFO - Loading model: ip-adapter-full-face_sd15 [852b9843]
2024-04-03 12:31:18,257 - ControlNet - INFO - Loaded state_dict from [D:\Stable Diffusion\stable-diffusion-webui-directml\models\ControlNet\ip-adapter-full-face_sd15.safetensors]
2024-04-03 12:31:18,561 - ControlNet - INFO - ControlNet model ip-adapter-full-face_sd15 [852b9843](ControlModelType.IPAdapter) loaded.
2024-04-03 12:31:18,568 - ControlNet - INFO - Using preprocessor: ip-adapter_face_id_plus
2024-04-03 12:31:18,569 - ControlNet - INFO - preprocessor resolution = 648
Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
find model: D:\Stable Diffusion\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\annotator\downloads\insightface\models\buffalo_l\1k3d68.onnx landmark_3d_68 ['None', 3, 192, 192] 0.0 1.0
Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
find model: D:\Stable Diffusion\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\annotator\downloads\insightface\models\buffalo_l\2d106det.onnx landmark_2d_106 ['None', 3, 192, 192] 0.0 1.0
Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
find model: D:\Stable Diffusion\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\annotator\downloads\insightface\models\buffalo_l\det_10g.onnx detection [1, 3, '?', '?'] 127.5 128.0
Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
find model: D:\Stable Diffusion\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\annotator\downloads\insightface\models\buffalo_l\genderage.onnx genderage ['None', 3, 96, 96] 0.0 1.0
Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
find model: D:\Stable Diffusion\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\annotator\downloads\insightface\models\buffalo_l\w600k_r50.onnx recognition ['None', 3, 112, 112] 127.5 127.5
set det-size: (640, 640)
D:\Stable Diffusion\stable-diffusion-webui-directml\SDenv\lib\site-packages\insightface\utils\transform.py:68: FutureWarning: `rcond` parameter will change to the default of machine precision times ``max(M, N)`` where M and N are the input matrix dimensions.
To use the future default and silence this warning we advise to pass `rcond=None`, to keep using the old, explicitly pass `rcond=-1`.
  P = np.linalg.lstsq(X_homo, Y)[0].T # Affine matrix. 3 x 4
2024-04-03 12:31:21,738 - ControlNet - INFO - Loading CLIP model on CPU.
*** Error running process: D:\Stable Diffusion\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\scripts\controlnet.py
    Traceback (most recent call last):
      File "D:\Stable Diffusion\stable-diffusion-webui-directml\modules\scripts.py", line 784, in process
        script.process(p, *script_args)
      File "D:\Stable Diffusion\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\scripts\controlnet.py", line 1245, in process
        self.controlnet_hack(p)
      File "D:\Stable Diffusion\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\scripts\controlnet.py", line 1230, in controlnet_hack
        self.controlnet_main_entry(p)
      File "D:\Stable Diffusion\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\scripts\controlnet.py", line 1189, in controlnet_main_entry      
        param.control_model.hook(
      File "D:\Stable Diffusion\stable-diffusion-webui-directml\SDenv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
        return func(*args, **kwargs)
      File "D:\Stable Diffusion\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\scripts\ipadapter\plugable_ipadapter.py", line 127, in hook      
        *[self.ipadapter.get_image_emb(o) for o in preprocessor_outputs]
      File "D:\Stable Diffusion\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\scripts\ipadapter\plugable_ipadapter.py", line 127, in <listcomp>        *[self.ipadapter.get_image_emb(o) for o in preprocessor_outputs]
      File "D:\Stable Diffusion\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\scripts\ipadapter\ipadapter_model.py", line 310, in get_image_emb        return self._get_image_embeds(preprocessor_output)
      File "D:\Stable Diffusion\stable-diffusion-webui-directml\SDenv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
        return func(*args, **kwargs)
      File "D:\Stable Diffusion\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\scripts\ipadapter\ipadapter_model.py", line 165, in _get_image_embeds
        clip_vision_output["hidden_states"][-2].to(
    TypeError: 'FaceIdPlusInput' object is not subscriptable

---
NegPiP enable, Positive:None,Negative:[5]
 27%|████████████████████████████████▊                                                                                          | 8/30 [00:27<01:15,  3.43s/it]

# Aborting Generation

Additional information

ip-adapter_face_id is producing IndexError:

2024-04-03 12:43:42,405 - ControlNet - INFO - unit_separate = False, style_align = False | 4/30 [00:10<00:54, 2.09s/it] 2024-04-03 12:43:42,406 - ControlNet - INFO - Loading model from cache: ip-adapter-plus-face_sd15 [7f7a633a] 2024-04-03 12:43:42,410 - ControlNet - INFO - Using preprocessor: ip-adapter_face_id 2024-04-03 12:43:42,410 - ControlNet - INFO - preprocessor resolution = 648 Error running process: D:\Stable Diffusion\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\scripts\controlnet.py Traceback (most recent call last): File "D:\Stable Diffusion\stable-diffusion-webui-directml\modules\scripts.py", line 784, in process script.process(p, script_args) File "D:\Stable Diffusion\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\scripts\controlnet.py", line 1245, in process self.controlnet_hack(p) File "D:\Stable Diffusion\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\scripts\controlnet.py", line 1230, in controlnet_hack self.controlnet_main_entry(p) File "D:\Stable Diffusion\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\scripts\controlnet.py", line 1189, in controlnet_main_entry
param.control_model.hook( File "D:\Stable Diffusion\stable-diffusion-webui-directml\SDenv\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context return func(*args, *kwargs) File "D:\Stable Diffusion\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\scripts\ipadapter\plugable_ipadapter.py", line 127, in hook
[self.ipadapter.get_image_emb(o) for o in preprocessor_outputs] File "D:\Stable Diffusion\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\scripts\ipadapter\plugable_ipadapter.py", line 127, in [self.ipadapter.get_image_emb(o) for o in preprocessor_outputs] File "D:\Stable Diffusion\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\scripts\ipadapter\ipadapter_model.py", line 310, in get_image_emb return self._get_image_embeds(preprocessor_output) File "D:\Stable Diffusion\stable-diffusion-webui-directml\SDenv\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context return func(args, kwargs) File "D:\Stable Diffusion\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\scripts\ipadapter\ipadapter_model.py", line 165, in _get_image_embeds clip_vision_output["hidden_states"][-2].to( IndexError: too many indices for tensor of dimension 2

huchenlei commented 7 months ago

You are using wrong preprocessor. Face plus model should not use faceid plus preprocessor. You can choose the ipadapter auto preprocessor now to auto pick the correct preprocessor now.