Intel Arc GPU + A1111-1.7.0-RC | Weird looking faces - Face Mask Correction/GFPGAN Not Working | FaceXLib issue

[HyunJae5463]

First, confirm

• [X] I have read the instruction carefully
• [X] I have searched the existing issues
• [X] I have updated the extension to the latest version

What happened?

Installed a fresh version of Stable Diffusion (A111). Everything worked fine before. With the new installation my faces look fuc*ed up though. When i try to use Face Mask Correction or change to GFPGAN the face is not swapping at all.

Steps to reproduce the problem

1. Put image into Reactor
2. Generate Image
3. Get fuc*ed up outputs

Sysinfo

Windows 11 Intel Arc A770 16GB IPEX Version of A1111

Relevant console log

Error when i try to use Face Mask Correction 

*** Error running postprocess_image: C:\AI\Automatic1111\stable-diffusion-webui-1.7.0-RC\extensions\sd-webui-reactor\scripts\reactor_faceswap.py
    Traceback (most recent call last):
      File "C:\AI\Automatic1111\stable-diffusion-webui-1.7.0-RC\modules\scripts.py", line 774, in postprocess_image
        script.postprocess_image(p, pp, *script_args)
      File "C:\AI\Automatic1111\stable-diffusion-webui-1.7.0-RC\extensions\sd-webui-reactor\scripts\reactor_faceswap.py", line 370, in postprocess_image
        result, output, swapped = swap_face(
      File "C:\AI\Automatic1111\stable-diffusion-webui-1.7.0-RC\extensions\sd-webui-reactor\scripts\reactor_swapper.py", line 578, in swap_face
        result_image, output, swapped = operate(source_img,target_img,target_img_orig,model,source_faces_index,faces_index,source_faces,target_faces,gender_source,gender_target,source_face,wrong_gender,source_age,source_gender,output,swapped,mask_face,entire_mask_image,enhancement_options)
      File "C:\AI\Automatic1111\stable-diffusion-webui-1.7.0-RC\extensions\sd-webui-reactor\scripts\reactor_swapper.py", line 672, in operate
        result = apply_face_mask(swapped_image=swapped_image,target_image=result,target_face=target_face,entire_mask_image=entire_mask_image)
      File "C:\AI\Automatic1111\stable-diffusion-webui-1.7.0-RC\extensions\sd-webui-reactor\reactor_modules\reactor_mask.py", line 60, in apply_face_mask
        mask = mask_generator.generate_mask(
      File "C:\AI\Automatic1111\stable-diffusion-webui-1.7.0-RC\extensions\sd-webui-reactor\scripts\reactor_inferencers\bisenet_mask_generator.py", line 48, in generate_mask
        face = self.mask_model(face_tensor)[0]
      File "C:\AI\Automatic1111\stable-diffusion-webui-1.7.0-RC\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "C:\AI\Automatic1111\stable-diffusion-webui-1.7.0-RC\venv\lib\site-packages\facexlib\parsing\bisenet.py", line 124, in forward
        feat_fuse = self.ffm(feat_sp, feat_cp8)
      File "C:\AI\Automatic1111\stable-diffusion-webui-1.7.0-RC\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "C:\AI\Automatic1111\stable-diffusion-webui-1.7.0-RC\venv\lib\site-packages\facexlib\parsing\bisenet.py", line 105, in forward
        feat_atten = torch.mul(feat, atten)
    RuntimeError: could not create a primitive descriptor for a reorder primitive

_____________________________________________________

Error when i try to use gfpgann
*** Error running postprocess_image: C:\AI\Automatic1111\stable-diffusion-webui-1.7.0-RC\extensions\sd-webui-reactor\scripts\reactor_faceswap.py
    Traceback (most recent call last):
      File "C:\AI\Automatic1111\stable-diffusion-webui-1.7.0-RC\modules\scripts.py", line 774, in postprocess_image
        script.postprocess_image(p, pp, *script_args)
      File "C:\AI\Automatic1111\stable-diffusion-webui-1.7.0-RC\extensions\sd-webui-reactor\scripts\reactor_faceswap.py", line 370, in postprocess_image
        result, output, swapped = swap_face(
      File "C:\AI\Automatic1111\stable-diffusion-webui-1.7.0-RC\extensions\sd-webui-reactor\scripts\reactor_swapper.py", line 578, in swap_face
        result_image, output, swapped = operate(source_img,target_img,target_img_orig,model,source_faces_index,faces_index,source_faces,target_faces,gender_source,gender_target,source_face,wrong_gender,source_age,source_gender,output,swapped,mask_face,entire_mask_image,enhancement_options)
      File "C:\AI\Automatic1111\stable-diffusion-webui-1.7.0-RC\extensions\sd-webui-reactor\scripts\reactor_swapper.py", line 711, in operate
        result_image = enhance_image(result_image, enhancement_options)
      File "C:\AI\Automatic1111\stable-diffusion-webui-1.7.0-RC\extensions\sd-webui-reactor\scripts\reactor_swapper.py", line 185, in enhance_image
        result_image = restore_face(result_image, enhancement_options)
      File "C:\AI\Automatic1111\stable-diffusion-webui-1.7.0-RC\extensions\sd-webui-reactor\scripts\reactor_swapper.py", line 146, in restore_face
        numpy_image = enhancement_options.face_restorer.restore(numpy_image)
      File "C:\AI\Automatic1111\stable-diffusion-webui-1.7.0-RC\modules\gfpgan_model.py", line 121, in restore
        return gfpgan_fix_faces(np_image)
      File "C:\AI\Automatic1111\stable-diffusion-webui-1.7.0-RC\modules\gfpgan_model.py", line 60, in gfpgan_fix_faces
        model = gfpgann()
      File "C:\AI\Automatic1111\stable-diffusion-webui-1.7.0-RC\modules\gfpgan_model.py", line 38, in gfpgann
        latest_file = max(gfp_models, key=os.path.getctime)
    ValueError: max() arg is an empty sequence

Additional information

No response

[Gourieff]

I will test it later, with 1.6.1 everything is ok Perhaps smth have been changed in 1.7.0-RC

[Gourieff]

Works wonderful with NVIDIA CUDA A1111 team added IPEX support for Intel What torch build do you have? pip show torch via activated VENV According to the log you provided we can see errors with facexlib and GFPGAN libs facexlib got torch error and if we google it - we can find that this error is connected with some intel instructions Do you encounter these errors when the "Face Mask Correction" option is off? Does GFPGAN work w/o "Face Mask Correction"?

[HyunJae5463]

My torch version is 2.1.1 When i have Face Mask Correction disabled i don't get errors, but then the face looks weird as shown in the screen above. GFPGAN does not work without Face Mask Correction either. Reactor works on the DirectML fork from lshqqytiger tho but since its slow af i switched to A1111 I also already went through the troubleshooting but it still happens so i guess its related to IPEX.

When i generate faces without Reactor or let Reactor save the original they are normal. Left Original Right after Reactor tried to face swap

Changing the model(checkpoint), face or Cross attention optimization doesnt change anything.

[Gourieff]

Could you please test GFPGAN separately? For the right image via Extras tab (with ReActor disabled). Will it be restored by GFPGAN?

[Gourieff]

And does it work with CodeFormer? Or same errors?

[HyunJae5463]

Normal generations without Reactor enabled work fine with both CodeFormers and GFPGAN.

Generations with Reactor: Restore Face with CodeFormer = Weird pixelated-looking outputs i have posted above Restore Face with GFPGAN = No Face Swap at all and Console Error posted above Restore Face with None selected = Same result as with CodeFormer Enable Face Correction = No Face Swap at all and Console Error posted above

And is this normal btw? That it uses the CPU? Got this message after reinstalling Reactor.

And just for reference: This is the result when i use the DirectML Fork but since Intel optimizes the Arc Drivers around IPEX and its MUCH faster i rather not use it. Face Swap works normally without any problems.

[Gourieff]

And is this normal btw? That it uses the CPU? Got this message after reinstalling Reactor.

Only two execution providers are now available in current version of ReActor: CPU and CUDA

Normal generations without Reactor enabled work fine with both CodeFormers and GFPGAN. Generations with Reactor: Restore Face with CodeFormer = Weird pixelated-looking outputs i have posted above Restore Face with GFPGAN = No Face Swap at all and Console Error posted above Restore Face with None selected = Same result as with CodeFormer Enable Face Correction = No Face Swap at all and Console Error posted above

So... We have to wait for A1111 1.7.0 will be released and will see whether this error will appear or not, because it seems that 1.7.0-RC can break some libs such as facexlib

[Pantonia4]

Same problem here. Tried every possible setting and launch command. I hope this can be fixed. IPEX is by far the fastest on Arc cards. I need 4 minutes to generate a 1000x1000 image on DirectML (if it even lets me and doesnt tell me im out of vram) and only like 30 on IPEX and OpenVino is just an unoptimized ram eating mess. Please please try to fix this.

[Gourieff]

@Pantonia4

Here's the test image - please put it to Extras and try to restore it with A1111's built-in CodeFormer or GFPGAN (and ReActor disabled)

[Pantonia4]

This is the outcome. It doesn't seem to change anything. I tried CodeFormer and GFPGAN and the result looks the exact same. There aren't any errors either.

[Gourieff]

Exactly! Thank you for this test! That's what I'm talking about in the thread https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/14224 Both CodeFormer and GFPGAN just don't work in 1.7.0-RC build when IPEX support is active and Intel Arc GPU is in use And ReActor has nothing to do with this

[llinlluo]

Installed a fresh version of Stable Diffusion (A1111), ReActor can't work.

Sysinfo Windows 11 NVIDIA GeForce RTX 3060 VRAM 12GB v1.7.0 Version of A1111

Relevant console log

[llinlluo]

--”NoneType“ Sysinfo Windows 11 NVIDIA GeForce RTX 3060 VRAM 12GB v1.6.0 Version of A1111

[Gourieff]

Installed a fresh version of Stable Diffusion (A1111), ReActor can't work.

Sysinfo Windows 11 NVIDIA GeForce RTX 3060 VRAM 12GB v1.7.0 Version of A1111

Relevant console log

On the 2nd screenshot you can see the messages about corrupted file (codeformer model), delete the model and download it again, or let A1111 to download it

[Gourieff]

--”NoneType“ Sysinfo Windows 11 NVIDIA GeForce RTX 3060 VRAM 12GB v1.6.0 Version of A1111

https://github.com/Gourieff/sd-webui-reactor#v-invalid_protobuf--load-model-from-modelsinsightfaceinswapper_128onnx-failedprotobuf-parsing-failed-or-attributeerror-nonetype-object-has-no-attribute-get-or-attributeerror-faceswapscript-object-has-no-attribute-save_original

Please be patient when scripts download models, don't interrupt the process otherwise you get corrupted files

[Gourieff]

@llinlluo and please create a new issue if you get stuck on smth and have no success in looking for a solution - don't use already existing threads with different topic This thread is about Intel Arc GPU + IPEX issues

[redrum-llik]

I am having the same kind of issue - face restore does not work out of the box for me. Using SD.Next (latest) with Arc A770, not passing --use-ipex on startup (for some reason it performs way worse with that key for me). Face Mask Correction does indeed cause RuntimeError: could not create a primitive descriptor for a reorder primitive, but switching to GFPGAN does not cause any extra errors in the console (and has no effect on the image output). Happy to provide any further details if anything could be helpful.

[HyunJae5463]

I am having the same kind of issue - face restore does not work out of the box for me. Using SD.Next (latest) with Arc A770, not passing --use-ipex on startup (for some reason it performs way worse with that key for me). Face Mask Correction does indeed cause RuntimeError: could not create a primitive descriptor for a reorder primitive, but switching to GFPGAN does not cause any extra errors in the console (and has no effect on the image output). Happy to provide any further details if anything could be helpful.

I've reported the issue here https://github.com/intel/intel-extension-for-pytorch/issues/487 more than a month ago but not much has happened yet. Maybe we are lucky and they fix it in next version.

[Dfafb]

So I had the error" RuntimeError: unexpected EOF, expected 7016529 more bytes. The file might be corrupted." on all CodeFormer or GFPGAN recovery faces and googled a simple solution for myself. Maybe it will help someone because the problem came out even on clean builds of AUTOMATIC 1.8.0 https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/8160#issuecomment-1582335527

delect stable-diffusion-webui/models/GFPGAN mkdir stable-diffusion-webui/models/GFPGAN run

[HyunJae5463]

So I had the error" RuntimeError: unexpected EOF, expected 7016529 more bytes. The file might be corrupted." on all CodeFormer or GFPGAN recovery faces and googled a simple solution for myself. Maybe it will help someone because the problem came out even on clean builds of AUTOMATIC 1.8.0 AUTOMATIC1111/stable-diffusion-webui#8160 (comment)

delect stable-diffusion-webui/models/GFPGAN mkdir stable-diffusion-webui/models/GFPGAN run

What are you supposed to do with that? Tried to run it in console it says "The term 'delect' is not recognized..."

[Dfafb]

What are you supposed to do with that?

delete all files from the GFPGAN folder. Then run the restoration face and it should automatically download the updated files