search

Gourieff / sd-webui-reactor

Fast and Simple Face Swap Extension for StableDiffusion WebUI (A1111 SD WebUI, SD WebUI Forge, SD.Next, Cagliostro)
GNU Affero General Public License v3.0
2.47k stars 264 forks source link

Running on VMs and Running on arbitary image shape. #160

Open akshay6893 opened 10 months ago

akshay6893 commented 10 months ago

First, confirm

What happened?

I faced 2 problems and here is the solution to them. Maybe these rectifications can be done in ReActor too -

FIRST

  1. ReActor uses insightface ( a python deep learing library). Insightface uses onxruntime or onxruntime-gpu.
  2. When you use ReActor (and onxruntime as a consequence) in a place where you have the physical GPU in some remote place, Like remote server, aws, collab etc. There is a problem - https://github.com/danielgatis/rembg/pull/529
  3. onxruntime automatically detects the number of threads to use for parallel processing on your CPU, based on your CPU architecture. But when you use Virtual Machines or virtual systems like servers etc, There is a VM abstraction layer between onxruntime and OS kernel/CPU. And onxruntime is not able to surpass it to know the details of your CPU, and can't know how to use threading. This gives off warnings and errors for threads.
  4. To solve this issue you have to follow the instructions here for setting the threads explicitly - https://github.com/microsoft/onnxruntime/issues/8313
  5. To set threads like this you have to change the insightface library itself. You can do this by Monkey patching. (ReActor does already use monkey patching in console_log_patch.py ). You will have to change the code in model_zoo.py file in insightface.
  6. This solved my problem with thread errors.

SECOND

  1. When a insight face gets images of different sizes, there are problems sometimes with the detection of face.
  2. This is solved in the ReActor code by halving det_size. But this didn't help in my case.
  3. Insightface suggests that your image shape should be divisible by 32. So do that before putting the src and target images in insightface.
  4. For me when the det_size was equal to the img shape, there was no problem in detection of face. So make it equal in the code.

I have made all the above changes in script dir. and it works fine (12 sec for one 576x320 image face swap on gpu) so if someone faces same problem. Let me know, I'll fork,commit.

Steps to reproduce the problem

  1. The first part of the problem only appears on VMs.
  2. The second on any arbitrary image shape.

Sysinfo

  1. firefox
  2. ubuntu 22.04
  3. rtx 3090
  4. version: v1.6.0  
  5. python: 3.9.4  
  6. torch: 1.12.0+cu113
  7. xformers: N/A
  8. gradio: 3.41.2

Relevant console log

Thread error looks like this - 

2023-05-18 08:24:58.694297027 [E:onnxruntime:Default, env.cc:251 ThreadMain] pthread_setaffinity_np failed for thread: 531, index: 6, mask: {7, 39, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2023-05-18 08:24:58.698247687 [E:onnxruntime:Default, env.cc:251 ThreadMain] pthread_setaffinity_np failed for thread: 532, index: 7, mask: {8, 40, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2023-05-18 08:24:58.702247091 [E:onnxruntime:Default, env.cc:251 ThreadMain] pthread_setaffinity_np failed for thread: 533, index: 8, mask: {9, 41, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.

Additional information

No response

Gourieff commented 10 months ago

Ok, will be implemented in future updates Or you can make a PR to the Evolve branch

adeng-wc commented 8 months ago

I also encountered the same problem, how to solve it? @akshay6893 can you make a PR to the Evolve branch？