[Issue]: Generation fails when Face scripts are used?

Biscuitbeard commented 6 months ago

Issue Description

New UI looks amazing, cheers for the great work. Unfortunately when I'm trying to use InstantID, Photomaker etc through the Face script, it's failing instantly with errors:

Any thoughts? This was installed through Stability Matrix if that changes anything. Cheers!

Version Platform Description

Python 3.10.11 on Windows
23:44:46-478085 INFO Version: app=sd.next updated=2024-05-29 hash=032018ab branch=master
url=https://github.com/vladmandic/automatic/tree/master
23:44:46-859084 INFO Platform: arch=AMD64 cpu=AMD64 Family 25 Model 33
Stepping 0, AuthenticAMD system=Windows
release=Windows-10-10.0.22631-SP0 python=3.10.11
23:44:46-867084 INFO nVidia CUDA toolkit detected: nvidia-smi present
23:44:48-854602 INFO Torch 2.3.0+cu121
23:44:48-873102 INFO Torch backend: nVidia CUDA 12.1 cuDNN 8801
23:44:48-876102 INFO Torch detected GPU: NVIDIA GeForce RTX 3090 VRAM 24576 Arch (8, 6) Cores 82
23:44:49-022108 INFO Extensions: disabled=[]
23:44:49-024109 INFO Extensions: enabled=['Lora', 'sd-extension-chainner', 'sd-extension-system-info',
'sd-webui-agent-scheduler', 'sdnext-modernui',
'stable-diffusion-webui-rembg'] extensions-builtin

Relevant log output

23:53:43-778879 INFO     Load model: time=20.10 load=16.50 move=3.59           
                         native=1024 {'ram': {'used': 2.74, 'total': 31.93},   
                         'gpu': {'used': 8.11, 'total': 24.0}, 'retries': 0,   
                         'oom': 0}                                             
23:53:43-798379 ERROR    Control pipeline failed: type=controlnet units=0      
                         error=cannot import name 'ValidationInfo' from        
                         'pydantic' (S:\STABLE DIFFUSION\Packages\SD.Next Web  
                         UI\venv\lib\site-packages\pydantic\__init__.cp310-win_
                         amd64.pyd)                                            
23:53:43-800380 ERROR    Control: ImportError                                  
┌───────────────────── Traceback (most recent call last) ─────────────────────┐
│ S:\STABLE DIFFUSION\Packages\SD.Next Web UI\modules\control\run.py:579 in c │
│                                                                             │
│   578 │   │   │   │   │   │                                                 │
│ > 579 │   │   │   │   │   │   processed = p.scripts.run(p, *p.script_args)  │
│   580 │   │   │   │   │   │   if processed is None:                         │
│                                                                             │
│ S:\STABLE DIFFUSION\Packages\SD.Next Web UI\modules\scripts.py:483 in run   │
│                                                                             │
│   482 │   │   parsed = p.per_script_args.get(script.title(), args[script.ar │
│ > 483 │   │   processed = script.run(p, *parsed)                            │
│   484 │   │   s.record(script.title())                                      │
│                                                                             │
│ S:\STABLE DIFFUSION\Packages\SD.Next Web UI\modules\face\__init__.py:117 in │
│                                                                             │
│   116 │   │   │   from modules.face.insightface import get_app              │
│ > 117 │   │   │   app = get_app('buffalo_l')                                │
│   118 │   │   │   from modules.face.faceid import face_id                   │
│                                                                             │
│ S:\STABLE DIFFUSION\Packages\SD.Next Web UI\modules\face\insightface.py:21  │
│                                                                             │
│   20 │   if insightface_app is None or mp_name != instightface_mp:          │
│ > 21 │   │   from insightface.app import FaceAnalysis                       │
│   22 │   │   import huggingface_hub as hf                                   │
│                                                                             │
│ S:\STABLE DIFFUSION\Packages\SD.Next Web UI\venv\lib\site-packages\insightf │
│                                                                             │
│   17 from . import utils                                                    │
│ > 18 from . import app                                                      │
│   19 from . import data                                                     │
│                                                                             │
│                           ... 1 frames hidden ...                           │
│                                                                             │
│ S:\STABLE DIFFUSION\Packages\SD.Next Web UI\venv\lib\site-packages\insightf │
│                                                                             │
│     3 import os.path as osp                                                 │
│ >   4 import albumentations as A                                            │
│     5 from albumentations.core.transforms_interface import ImageOnlyTransfo │
│                                                                             │
│ S:\STABLE DIFFUSION\Packages\SD.Next Web UI\venv\lib\site-packages\albument │
│                                                                             │
│    4                                                                        │
│ >  5 from .augmentations import *                                           │
│    6 from .core.composition import *                                        │
│                                                                             │
│ S:\STABLE DIFFUSION\Packages\SD.Next Web UI\venv\lib\site-packages\albument │
│                                                                             │
│    1 from .blur.functional import *                                         │
│ >  2 from .blur.transforms import *                                         │
│    3 from .crops.functional import *                                        │
│                                                                             │
│ S:\STABLE DIFFUSION\Packages\SD.Next Web UI\venv\lib\site-packages\albument │
│                                                                             │
│   1 from .functional import *                                               │
│ > 2 from .transforms import *                                               │
│   3                                                                         │
│                                                                             │
│ S:\STABLE DIFFUSION\Packages\SD.Next Web UI\venv\lib\site-packages\albument │
│                                                                             │
│     6 import numpy as np                                                    │
│ >   7 from pydantic import Field, ValidationInfo, field_validator, model_va │
│     8 from typing_extensions import Self                                    │
└─────────────────────────────────────────────────────────────────────────────┘
ImportError: cannot import name 'ValidationInfo' from 'pydantic' (S:\STABLE DIFFUSION\Packages\SD.Next Web UI\venv\lib\site-packages\pydantic\__init__.cp310-win_amd64.pyd)

Backend

Diffusers

Branch

Master

Model

SD-XL

Acknowledgements

[X] I have read the above and searched for existing issues
[X] I confirm that this is classified correctly and its not an extension issue

vladmandic commented 6 months ago

this is a package dependency version issue, looks like something got "upgraded" to bad version outside of sdnext control.

I'll check. once I find the culprit, it will be a easy fix.

brknsoul commented 6 months ago

This error is caused by insightface installing modules that are "too new". We'll counter this by manually installing insightface, and then specific versions of these modules.

Open command prompt in stable-diffusion-webui-amdgpu-forge folder.

Run these commands one after the other;

venv\Scripts\activate
pip install insightface
pip install albumentations==1.4.3
pip install pydantic==1.10.15

Close command prompt and run webui-user.bat again.

Biscuitbeard commented 6 months ago

Man you guys work fast! Thanks Vlad, I've tried brknsoul's tip (looks like Pydantic was instlaled with a much higher version, 2.7 or something). However I can't get regular text to image to work any more after trying the SVD script in my last session.

It seems like it's somehow stuck in the SVD pipeline? If i run img2img it generates but then fails at the end saying _required positional argument: 'numframes'

If i try to run txt2img it simply fails with the following:

01:40:12-854513 WARNING Pipeline class change failed:
type=DiffusersTaskType.TEXT_2_IMAGE
pipeline=StableVideoDiffusionPipeline AutoPipeline
can't find a pipeline linked to
StableVideoDiffusionPipeline for None
01:40:12-883512 INFO Base: class=StableVideoDiffusionPipeline
01:40:12-923513 ERROR Control pipeline failed: type=controlnet units=0
error=StableVideoDiffusionPipeline.call() missing 1 required positional argument: 'image'
01:40:12-925514 ERROR Control: TypeError
┌───────────────────── Traceback (most recent call last) ─────────────────────┐ │ S:\STABLE DIFFUSION\Packages\SD.Next Web UI\modules\control\run.py:581 in c │ │ │ │ 580 │ │ │ │ │ │ if processed is None: │ │ > 581 │ │ │ │ │ │ │ processed: processing.Processed = process │ │ 582 │ │ │ │ │ │ else: │ │ │ │ S:\STABLE DIFFUSION\Packages\SD.Next Web UI\modules\processing.py:192 in pr │ │ │ │ 191 │ │ │ with context_hypertile_vae(p), context_hypertile_unet(p): │ │ > 192 │ │ │ │ processed = process_images_inner(p) │ │ 193 │ │ │ │ S:\STABLE DIFFUSION\Packages\SD.Next Web UI\modules\processing.py:312 in pr │ │ │ │ 311 │ │ │ │ │ from modules.processingdiffusers import process │ │ > 312 │ │ │ │ │ x_samples_ddim = process_diffusers(p) │ │ 313 │ │ │ │ else: │ │ │ │ S:\STABLE DIFFUSION\Packages\SD.Next Web UI\modules\processing_diffusers.py │ │ │ │ 117 │ │ # base_args['image'] = set_latents(p) │ │ > 118 │ │ output = shared.sd_model(*base_args) # pylint: disable=not-c │ │ 119 │ │ if isinstance(output, dict): │ │ │ │ S:\STABLE DIFFUSION\Packages\SD.Next Web UI\venv\lib\site-packages\torch\ut │ │ │ │ 114 │ │ with ctx_factory(): │ │ > 115 │ │ │ return func(args, **kwargs) │ │ 116 │ └─────────────────────────────────────────────────────────────────────────────┘ TypeError: StableVideoDiffusionPipeline.call() missing 1 required positional argument: 'image'

Might just be that a Stability Matrix update will be required to get the right requirements etc, but letting you know in case it's an issue you might want to fix. Thanks again!

brknsoul commented 6 months ago

That sounds like a separate problem and should be in its own issue report.

Biscuitbeard commented 6 months ago

Good point, if they don't look related I might have to raise it! Will try a fresh install for now though, I don't generally use SVD anyway 👍

vladmandic commented 6 months ago

it sounds like you still have svd model loaded even if you're not using it - you need to select some other model to load. It's just thar SVD is too big, so can't "store" normal model and then restore.

Biscuitbeard commented 6 months ago

Yeah, I figured it out - sorry for not realising sooner! now I'm getting a new error with the face models:

02:23:54-086844 ERROR Control pipeline failed: type=controlnet units=0
error='NoneType' object is not subscriptable
02:23:54-088844 ERROR Control: TypeError
┌───────────────────── Traceback (most recent call last) ─────────────────────┐ │ S:\STABLE DIFFUSION\Packages\SD.Next Web UI\modules\control\run.py:579 in c │ │ │ │ 578 │ │ │ │ │ │ │ │ > 579 │ │ │ │ │ │ processed = p.scripts.run(p, p.script_args) │ │ 580 │ │ │ │ │ │ if processed is None: │ │ │ │ S:\STABLE DIFFUSION\Packages\SD.Next Web UI\modules\scripts.py:483 in run │ │ │ │ 482 │ │ parsed = p.per_script_args.get(script.title(), args[script.ar │ │ > 483 │ │ processed = script.run(p, parsed) │ │ 484 │ │ s.record(script.title()) │ │ │ │ S:\STABLE DIFFUSION\Packages\SD.Next Web UI\modules\face__init__.py:119 in │ │ │ │ 118 │ │ │ from modules.face.faceid import face_id │ │ > 119 │ │ │ processed_images = face_id(p, app=app, source_images=inpu │ │ 120 │ │ │ processed = processing.Processed(p, images_list=processed │ │ │ │ S:\STABLE DIFFUSION\Packages\SD.Next Web UI\modules\face\faceid.py:206 in f │ │ │ │ 205 │ │ │ │ p.iteration = i │ │ > 206 │ │ │ │ p.prompts = p.all_prompts[i p.batch_size:(i + 1) │ │ 207 │ │ │ │ p.negative_prompts = p.all_negative_prompts[i * p.bat │ └─────────────────────────────────────────────────────────────────────────────┘ TypeError: 'NoneType' object is not subscriptable``

Sorry for taking up your time here with tech support - I hope it's something useful to other users 😆

vladmandic commented 6 months ago

both issues are resolved:

faceid invalid prompt handling
insightface installer with correct dependencies

Biscuitbeard commented 6 months ago

Cheers man!

Even on the latest build I'm still getting 'object is not subscriptable' errors using Photomaker, is it something wrong at my end?

16:24:01-380293 INFO Load model: time=20.57 load=16.73 move=3.84
native=1024 {'ram': {'used': 3.55, 'total': 31.93},
'gpu': {'used': 8.12, 'total': 24.0}, 'retries': 0,
'oom': 0}
16:24:01-389308 ERROR Control pipeline failed: type=controlnet units=0
error='NoneType' object is not subscriptable
16:24:01-390809 ERROR Control: TypeError
┌───────────────────── Traceback (most recent call last) ─────────────────────┐ │ S:\STABLE DIFFUSION\Packages\SD.Next Web UI\modules\control\run.py:579 in c │ │ │ │ 578 │ │ │ │ │ │ │ │ > 579 │ │ │ │ │ │ processed = p.scripts.run(p, p.script_args) │ │ 580 │ │ │ │ │ │ if processed is None: │ │ │ │ S:\STABLE DIFFUSION\Packages\SD.Next Web UI\modules\scripts.py:483 in run │ │ │ │ 482 │ │ parsed = p.per_script_args.get(script.title(), args[script.ar │ │ > 483 │ │ processed = script.run(p, parsed) │ │ 484 │ │ s.record(script.title()) │ │ │ │ S:\STABLE DIFFUSION\Packages\SD.Next Web UI\modules\face__init__.py:123 in │ │ │ │ 122 │ │ │ from modules.face.photomaker import photo_maker │ │ > 123 │ │ │ processed = photo_maker(p, input_images=input_images, tri │ │ 124 │ │ elif mode == 'InstantID': │ │ │ │ S:\STABLE DIFFUSION\Packages\SD.Next Web UI\modules\face\photomaker.py:21 i │ │ │ │ 20 │ trigger_ids = shared.sd_model.tokenizer.encode(trigger) + shared.s │ │ > 21 │ prompt_ids1 = shared.sd_model.tokenizer.encode(p.all_prompts[0]) │ │ 22 │ prompt_ids2 = shared.sd_model.tokenizer_2.encode(p.all_prompts[0]) │ └─────────────────────────────────────────────────────────────────────────────┘ TypeError: 'NoneType' object is not subscriptable``

EDIT: instantID now works though!

vladmandic commented 5 months ago

@Biscuitbeard can you try latest update, should be fixed now?

vladmandic / automatic