[Extension]: facexlib does now work with ipex

HyunJae5463 commented 7 months ago

Issue Description

When using IPEX, Face Restoration is not working. This issue was also already reported on the Automatic1111 version when using IPEX and the creator of the Reactor Extenstion also said that Reactor (FaceSwap Extension) is not working with IPEX for that reason. Apparently, facexlib is the root of this issue. Everything works fine when using DirectML and OpenVINO

Steps to reproduce the problem

Go to Process
Add any image with a face
Use either GFPGAN or CodeFormer
Output is 1:1 the same as the Input

Or when using Reactor

Enable Reactor and use any image of a face you want to swap.
Generate image via text2image.
The output looks blurred and pixelated

Version Platform Description

Win 11 Ryzen 5 5600x Intel Arc A770 16GB 32GB Ram

Browser: Edge

Using VENV: C:\AI\automatic\venv 12:34:18-162996 INFO Starting SD.Next 12:34:18-165997 INFO Logger: file="C:\AI\automatic\sdnext.log" level=INFO size=95319 mode=append 12:34:18-166996 INFO Python 3.10.11 on Windows 12:34:26-571273 INFO Version: app=sd.next updated=2023-12-17 hash=83785628 url=https://github.com/vladmandic/automatic/tree/master 12:34:31-059438 INFO Platform: arch=AMD64 cpu=AMD64 Family 25 Model 33 Stepping 2, AuthenticAMD system=Windows release=Windows-10-10.0.22631-SP0 python=3.10.11 12:34:31-061438 INFO Intel OneAPI Toolkit detected 12:34:35-285973 INFO Extensions: disabled=[] 12:34:35-286973 INFO Extensions: enabled=['Lora', 'sd-extension-chainner', 'sd-extension-system-info', 'sd-webui-agent-scheduler', 'sd-webui-controlnet', 'stable-diffusion-webui-images-browser', 'stable-diffusion-webui-rembg'] extensions-builtin 12:34:35-293973 INFO Extensions: enabled=['sd-webui-reactor'] extensions 12:34:35-295973 INFO Startup: quick launch 12:34:35-296973 INFO Verifying requirements 12:34:35-307973 INFO Verifying packages 12:34:35-309973 INFO Extensions: disabled=[] 12:34:35-310972 INFO Extensions: enabled=['Lora', 'sd-extension-chainner', 'sd-extension-system-info', 'sd-webui-agent-scheduler', 'sd-webui-controlnet', 'stable-diffusion-webui-images-browser', 'stable-diffusion-webui-rembg'] extensions-builtin 12:34:35-315973 INFO Extensions: enabled=['sd-webui-reactor'] extensions 12:34:35-321974 INFO Extension preload: {'extensions-builtin': 0.0, 'extensions': 0.0} 12:34:35-323973 INFO Command line args: ['--use-ipex', '--medvram'] medvram=True use_ipex=True C:\AI\automatic\venv\lib\site-packages\torchvision\io\image.py:13: UserWarning: Failed to load image Python extension: ''If you don't plan on using image functionality from torchvision.io, you can ignore this warning. Otherwise, there might be something wrong with your environment. Did you have libjpeg or libpng installed before building torchvision from source? warn( 12:34:51-249020 INFO Load packages: torch=2.0.0a0+gite9ebda2 diffusers=0.24.0 gradio=3.43.2 12:34:51-988615 INFO Engine: backend=Backend.ORIGINAL compute=ipex mode=no_grad device=xpu cross-optimization="Scaled-Dot-Product" 12:34:51-992614 INFO Device: device=Intel(R) Arc(TM) A770 Graphics n=1 ipex=2.0.110+gitc6ea20b 12:34:59-325102 INFO Available VAEs: path="models\VAE" items=1 12:34:59-327102 INFO Disabling uncompatible extensions: backend=Backend.ORIGINAL [] 12:34:59-331110 INFO Available models: path="models\Stable-diffusion" items=1 time=0.00 12:35:01-781923 INFO Extension: script='extensions-builtin\sd-webui-agent-scheduler\scripts\task_scheduler.py' Using sqlite file: extensions-builtin\sd-webui-agent-scheduler\task_scheduler.sqlite3 12:35:02-033959 INFO Extension: script='extensions-builtin\sd-webui-controlnet\scripts\api.py' ControlNet preprocessor location: C:\AI\automatic\extensions-builtin\sd-webui-controlnet\annotator\downloads 12:35:02-297491 INFO Extension: script='extensions-builtin\sd-webui-controlnet\scripts\controlnet.py' Warning: ControlNet failed to load SGM - will use LDM instead. 12:35:02-306499 INFO Extension: script='extensions-builtin\sd-webui-controlnet\scripts\hook.py' Warning: ControlNet failed to load SGM - will use LDM instead. 12:35:04-899818 INFO Extensions time: 4.80 { Lora=1.23 sd-extension-chainner=0.06 sd-webui-agent-scheduler=0.37 sd-webui-controlnet=0.53 stable-diffusion-webui-images-browser=0.07 stable-diffusion-webui-rembg=1.67 sd-webui-reactor=0.84 } 12:35:05-042334 INFO Load UI theme: name="gradio/default" style=Auto base=base.css 12:35:05-044842 WARNING Using Gradio default theme which is not optimized for SD.Next 12:35:07-552656 INFO Local URL: http://127.0.0.1:7860/ 12:35:07-554657 INFO Initializing middleware 12:35:07-739177 INFO [AgentScheduler] Task queue is empty 12:35:07-742175 INFO [AgentScheduler] Registering APIs 12:35:08-160739 INFO Startup time: 32.83 { torch=14.16 gradio=1.71 diffusers=0.05 libraries=8.07 extensions=4.80 face-restore=0.76 ui-extra-networks=0.13 ui-txt2img=0.06 ui-img2img=0.08 ui-settings=0.50 ui-extensions=1.37 ui-defaults=0.07 launch=0.32 api=0.10 app-started=0.50 } 12:35:32-280694 INFO MOTD: N/A 12:35:36-340343 INFO Browser session: user=None client=127.0.0.1 agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36 Edg/120.0.0.0 12:36:07-811323 INFO Select: model="Cosplay2 [6b144dc3ac]" Loading model: C:\AI\automatic\models\Stable-diffusion\Cosplay2.safetensors ━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/2.3 GB -:--:-- 12:36:09-302034 INFO Setting Torch parameters: device=xpu dtype=torch.bfloat16 vae=torch.bfloat16 unet=torch.bfloat16 context=no_grad fp16=False bf16=True 12:36:16-095003 INFO LDM: LatentDiffusion: mode=eps 12:36:16-096001 INFO LDM: DiffusionWrapper params=859.52M 12:36:16-097001 INFO Autodetect: model="Stable Diffusion" class=StableDiffusionPipeline

Backend

Original

Branch

Master

Model

SD 1.5

Acknowledgements

[X] I have read the above and searched for existing issues
[X] I confirm that this is classified correctly and its not an extension issue

Disty0 commented 7 months ago

Report this to IPEX or ONNX Runtime.

Also extension author should add OpenVINO Execution Provider for ONNX if you want GPU support. OpenVINO is the only way to run ONNX on ARC. Fun fact: OpenVINO is universal, aka. supports AMD GPUs on Windows too.

HyunJae5463 commented 7 months ago

Report this to IPEX or ONNX Runtime.

Also extension author should add OpenVINO Execution Provider for ONNX if you want GPU support. OpenVINO is the only way to run ONNX on ARC. Fun fact: OpenVINO is universal, aka. supports AMD GPUs on Windows too.

Thank you. I saw that IPEX released a new version 2 days ago. Is there any way for me to update to that version and see if it fixes the issue? Or is there something that has to be done on your end?

I tried to simply copy&paste its content into venv\Lib\site-packages\intel_extension_for_pytorch and Face Restoration actually started to work but it started using my CPU instead of my GPU. Now i'm not sure however if the new version fixed it or that SD used my CPU instead of GPU.

I also can't find a requirement file for ipex to change the version unless im blind

Disty0 commented 7 months ago

Switch to dev branch and run the webui with --reinstall.

HyunJae5463 commented 7 months ago

Switch to dev branch and run the webui with --reinstall.

I did, now its not starting anymore at all

╭───────────────────────────────────────── Traceback (most recent call last) ──────────────────────────────────────────╮
│ C:\AI\Test\automatic\launch.py:222 in <module>                                                                       │
│                                                                                                                      │
│   221                                                                                                                │
│ ❱ 222     uv, instance = start_server(immediate=True, server=None)                                                   │
│   223     while True:                                                                                                │
│                                                                                                                      │
│ C:\AI\Test\automatic\launch.py:152 in start_server                                                                   │
│                                                                                                                      │
│   151     get_custom_args()                                                                                          │
│ ❱ 152     module_spec.loader.exec_module(server)                                                                     │
│   153     uvicorn = None                                                                                             │
│ in exec_module:883                                                                                                   │
│ in _call_with_frames_removed:241                                                                                     │
│                                                                                                                      │
│ C:\AI\Test\automatic\webui.py:11 in <module>                                                                         │
│                                                                                                                      │
│    10 from threading import Thread                                                                                   │
│ ❱  11 import modules.loader                                                                                          │
│    12 import torch # pylint: disable=wrong-import-order                                                              │
│                                                                                                                      │
│ C:\AI\Test\automatic\modules\loader.py:16 in <module>                                                                │
│                                                                                                                      │
│   15                                                                                                                 │
│ ❱ 16 import torch # pylint: disable=C0411                                                                            │
│   17                                                                                                                 │
│                                                                                                                      │
│ C:\AI\Test\automatic\venv\lib\site-packages\torch\__init__.py:139 in <module>                                        │
│                                                                                                                      │
│    138                 err.strerror += f' Error loading "{dll}" or one of its dependencies.'                         │
│ ❱  139                 raise err                                                                                     │
│    140                                                                                                               │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
OSError: [WinError 126] The specified module could not be found. Error loading
"C:\AI\Test\automatic\venv\lib\site-packages\torch\lib\backend_with_compiler.dll" or one of its dependencies.
Press any key to continue . . .

Disty0 commented 7 months ago

Welp, Windows is still missing libuv i guess. Will take care of this.

Disty0 commented 7 months ago

Updated the installer with @Nuullll's packages. Update the webui and do a reinstall.

HyunJae5463 commented 7 months ago

Thanks for your time. Unfortunately, Face Restoration is still not working. So i guess it only works when you use your CPU but not when you use your GPU for some reason I will report this on the IPEX Github page and hope that they have a solution for it.

vladmandic / automatic