[Bug]: ip-adapter_face_id with Multi-Inputs not working

QunBB commented 8 months ago

Is there an existing issue for this?

[X] I have searched the existing issues and checked the recent builds/commits of both this extension and the webui

What happened?

When i use ip-adapter_face_id and put six images on Multi-Inputs, I think it only use just one input image. It should cause by ip-adapter_face_id preprocessor, because it will also happend when using other model like ip-adapter-faceid_sd15.

Steps to reproduce the problem

Enable IP-Adapter
Select ip-adapter_face_id as preprocessor and ip-adapter-faceid-portrait_sd15 as model

What should have happened?

It should use six images of all my inputs and handle the preprocessor six times.

Commit where the problem happens

webui: controlnet: v1.1.435

What browsers do you use to access the UI ?

Google Chrome

Command Line Arguments

python launch.py --listen --api --precision full --no-half --xformers --theme dark --enable-insecure-extension-access --log-startup --ad-no-huggingface --disable-safe-unpickle

List of enabled extensions

Console logs

The logs when using `ip-adapter_face_id` as preprocessor:

Total progress: 100%|██████████| 19/19 [00:03<00:00,  4.81it/s]
2024-01-23 16:30:48,429 - ControlNet - INFO - unit_separate = False, style_align = False
2024-01-23 16:30:48,619 - ControlNet - INFO - Loading model: ip-adapter-faceid-portrait_sd15 [b2609049]
2024-01-23 16:30:48,639 - ControlNet - INFO - Loaded state_dict from [/data/ai/stable-diffusion-webui-160/extensions/sd-webui-controlnet/models/ip-adapter-faceid-portrait_sd15.bin]
2024-01-23 16:30:48,726 - ControlNet - INFO - ControlNet model ip-adapter-faceid-portrait_sd15 [b2609049] loaded.
2024-01-23 16:30:48,759 - ControlNet - INFO - Using preprocessor: ip-adapter_face_id
2024-01-23 16:30:48,760 - ControlNet - INFO - preprocessor resolution = 512
2024-01-23 16:30:48,778 - ControlNet - INFO - ControlNet Hooked - Time = 0.3591439723968506
100%|██████████| 19/19 [00:03<00:00,  5.67it/s]
Total progress: 100%|██████████| 19/19 [00:04<00:00,  4.64it/s]
Total progress: 100%|██████████| 19/19 [00:04<00:00,  5.72it/s]

The logs when using other preprocessor:

2024-01-23 16:15:56,203 - ControlNet - INFO - unit_separate = False, style_align = False
2024-01-23 16:15:56,434 - ControlNet - INFO - Loading model: ip-adapter-faceid-plusv2_sd15 [6e14fc1a]
2024-01-23 16:15:56,476 - ControlNet - INFO - Loaded state_dict from [/data/ai/stable-diffusion-webui-160/extensions/sd-webui-controlnet/models/ip-adapter-faceid-plusv2_sd15.bin]
2024-01-23 16:15:56,477 - ControlNet - INFO - IP-Adapter faceid plus v2 detected.
2024-01-23 16:15:56,693 - ControlNet - INFO - ControlNet model ip-adapter-faceid-plusv2_sd15 [6e14fc1a] loaded.
2024-01-23 16:15:56,697 - ControlNet - INFO - Using preprocessor: ip-adapter_face_id_plus
2024-01-23 16:15:56,697 - ControlNet - INFO - preprocessor resolution = 512
2024-01-23 16:15:56,869 - ControlNet - INFO - Loading model: ip-adapter-faceid-plusv2_sd15 [6e14fc1a]
2024-01-23 16:15:56,913 - ControlNet - INFO - Loaded state_dict from [/data/ai/stable-diffusion-webui-160/extensions/sd-webui-controlnet/models/ip-adapter-faceid-plusv2_sd15.bin]
2024-01-23 16:15:56,913 - ControlNet - INFO - IP-Adapter faceid plus v2 detected.
2024-01-23 16:15:57,134 - ControlNet - INFO - ControlNet model ip-adapter-faceid-plusv2_sd15 [6e14fc1a] loaded.
2024-01-23 16:15:57,141 - ControlNet - INFO - Using preprocessor: ip-adapter_face_id_plus
2024-01-23 16:15:57,141 - ControlNet - INFO - preprocessor resolution = 512
2024-01-23 16:15:57,305 - ControlNet - INFO - Loading model: ip-adapter-faceid-plusv2_sd15 [6e14fc1a]
2024-01-23 16:15:57,349 - ControlNet - INFO - Loaded state_dict from [/data/ai/stable-diffusion-webui-160/extensions/sd-webui-controlnet/models/ip-adapter-faceid-plusv2_sd15.bin]
2024-01-23 16:15:57,349 - ControlNet - INFO - IP-Adapter faceid plus v2 detected.
2024-01-23 16:15:57,570 - ControlNet - INFO - ControlNet model ip-adapter-faceid-plusv2_sd15 [6e14fc1a] loaded.
2024-01-23 16:15:57,576 - ControlNet - INFO - Using preprocessor: ip-adapter_face_id_plus
2024-01-23 16:15:57,576 - ControlNet - INFO - preprocessor resolution = 512
2024-01-23 16:15:57,749 - ControlNet - INFO - Loading model: ip-adapter-faceid-plusv2_sd15 [6e14fc1a]
2024-01-23 16:15:57,792 - ControlNet - INFO - Loaded state_dict from [/data/ai/stable-diffusion-webui-160/extensions/sd-webui-controlnet/models/ip-adapter-faceid-plusv2_sd15.bin]
2024-01-23 16:15:57,792 - ControlNet - INFO - IP-Adapter faceid plus v2 detected.
2024-01-23 16:15:58,017 - ControlNet - INFO - ControlNet model ip-adapter-faceid-plusv2_sd15 [6e14fc1a] loaded.
2024-01-23 16:15:58,023 - ControlNet - INFO - Using preprocessor: ip-adapter_face_id_plus
2024-01-23 16:15:58,023 - ControlNet - INFO - preprocessor resolution = 512
2024-01-23 16:15:58,210 - ControlNet - INFO - Loading model: ip-adapter-faceid-plusv2_sd15 [6e14fc1a]
2024-01-23 16:15:58,256 - ControlNet - INFO - Loaded state_dict from [/data/ai/stable-diffusion-webui-160/extensions/sd-webui-controlnet/models/ip-adapter-faceid-plusv2_sd15.bin]
2024-01-23 16:15:58,256 - ControlNet - INFO - IP-Adapter faceid plus v2 detected.
2024-01-23 16:15:58,479 - ControlNet - INFO - ControlNet model ip-adapter-faceid-plusv2_sd15 [6e14fc1a] loaded.
2024-01-23 16:15:58,487 - ControlNet - INFO - Using preprocessor: ip-adapter_face_id_plus
2024-01-23 16:15:58,487 - ControlNet - INFO - preprocessor resolution = 512
2024-01-23 16:15:58,654 - ControlNet - INFO - Loading model: ip-adapter-faceid-plusv2_sd15 [6e14fc1a]
2024-01-23 16:15:58,699 - ControlNet - INFO - Loaded state_dict from [/data/ai/stable-diffusion-webui-160/extensions/sd-webui-controlnet/models/ip-adapter-faceid-plusv2_sd15.bin]
2024-01-23 16:15:58,699 - ControlNet - INFO - IP-Adapter faceid plus v2 detected.
2024-01-23 16:15:58,897 - ControlNet - INFO - ControlNet model ip-adapter-faceid-plusv2_sd15 [6e14fc1a] loaded.
2024-01-23 16:15:58,926 - ControlNet - INFO - Using preprocessor: ip-adapter_face_id_plus
2024-01-23 16:15:58,926 - ControlNet - INFO - preprocessor resolution = 512
2024-01-23 16:15:59,039 - ControlNet - INFO - ControlNet Hooked - Time = 2.8418569564819336
100%|██████████| 19/19 [00:03<00:00,  5.23it/s]
Total progress: 100%|██████████| 19/19 [00:04<00:00,  4.43it/s]

Additional information

It worked in v1.1.434!

huchenlei commented 8 months ago

See https://github.com/Mikubill/sd-webui-controlnet/pull/2539. Now you don't need to calculate weight based on number of images. Now you can just use weight = 1.

QunBB commented 8 months ago

@huchenlei I think the main problem isn't the weight. You could see the Console logs above: When i use ip-adapter_face_id_plus as preprocessor, the logs show the process six times. But only once when i using ip-adapter_face_id. By the way, I used the same six images input.

huchenlei commented 8 months ago

Real multi inputs is only implemented for faceid preprocessor now. So it is expected you see only one unit in log. For faceid plus, the implementation is still using multiple units. I will update all ipadapter preprocessor to accept multiple inputs later today.

huchenlei commented 8 months ago

Btw, the log you provided does not match with the setting in the screenshot. You are using faceid portrait in screenshot but faceid plus v2 in the log.

QunBB commented 8 months ago

@huchenlei Thanks for your reply. I have understood it. Actually the log i provided includes two part, about faceid portrait and faceid plus. But the reason why i do have the confusion is upgrading the version to v1.1.435 from v1.1.434, which lead to the significantly differentperformance of faceid portrait. I will show the details tomorrow.

QunBB commented 8 months ago

@huchenlei I feel that the results on v1.1.434 looks more like Yang Mi(杨幂) on my own. And on v1.1.435, the results with six images looks the same as the results with single image.

v1.1.435 commit id: 7fb40bf803d05cde9ca9db8b1c6b8ee3c5efb352 v1.1.434 commit id: 81b5dde69a88a033f3bb2c086d04ca7b6f91272e

v1.1.435: Put six images on Multi-Inputs, the result is as follow