"RuntimeError: Found at least two devices"

tailgunnerx commented 9 months ago

When using PhotoMaker for the first time:

RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!

Per ChatGPT4:

The error message indicates that there's a device mismatch between the tensors involved in the operation. Specifically, the error states:

vbnet

RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!

This means that part of your computation is happening on the GPU (cuda:0) and another part on the CPU, which is not allowed during tensor operations in PyTorch. All tensors involved in an operation must be on the same device.

To resolve this issue, ensure that all tensors and models involved in the computation are moved to the same device. You can explicitly move tensors to a device using .to(device) method where device is either "cpu" or a specific CUDA device like "cuda:0".

For example, if you have a tensor tensor and a model model, and you want to perform operations on the GPU, you can do:

python

device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu") tensor = tensor.to(device) model = model.to(device)

You should apply this to all tensors and models involved in the operations leading to the error. If some tensors are generated within a function or a library call, check the documentation for those functions or libraries to see if there's a way to specify the device for their internal computations.

In your specific case, it looks like the error is happening within the transformers library during the forward pass of a GPT-2 model. If you're using pre-built functions or classes from a library, ensure that the model and all input tensors are moved to the same device before calling such functions. If the library provides a way to set the device for its internal operations, make sure to use it.

machineminded commented 9 months ago

Does it give a stacktrace of this error?

tailgunnerx commented 9 months ago

Does it give a stacktrace of this error?

not sure where to find this information

machineminded commented 9 months ago

Does it give a stacktrace of this error?

not sure where to find this information

Stack trace looks like this in python. I found a similar error with Fooocus on Reddit. What were you trying to do when this error was returned?

### more log entries occurred before this line

File "E:\FOOOCUS\Fooocus_win64_2-1-831\python_embeded\lib\site-packages\torch\nn\modules\[module.py](https://module.py/)", line 1518, in _wrapped_call_impl

return self._call_impl(*args, **kwargs)

File "E:\FOOOCUS\Fooocus_win64_2-1-831\python_embeded\lib\site-packages\torch\nn\modules\[module.py](https://module.py/)", line 1527, in _call_impl

return forward_call(*args, **kwargs)

File "E:\FOOOCUS\Fooocus_win64_2-1-831\python_embeded\lib\site-packages\transformers\models\gpt2\modeling_gpt2.py", line 332, in forward

attn_output, attn_weights = self._attn(query, key, value, attention_mask, head_mask)

File "E:\FOOOCUS\Fooocus_win64_2-1-831\python_embeded\lib\site-packages\transformers\models\gpt2\modeling_gpt2.py", line 203, in _attn

attn_weights = torch.where(causal_mask, attn_weights.to(attn_weights.dtype), mask_value)

RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!

machineminded commented 9 months ago

I noticed that I'm not getting this error if I run launch.py from cmd. What would be the difference vs. executing launch.py by clicking on it?

I think the difference would be that one is using the virtual environment venv and the other would be using the python environment on your system. If your system environment is working then that is okay too. Do you have other UIs installed like AUTOMATIC1111 or ComfyUI?

tailgunnerx commented 9 months ago

I noticed that I'm not getting this error if I run launch.py from cmd. What would be the difference vs. executing launch.py by clicking on it?

I think the difference would be that one is using the virtual environment venv and the other would be using the python environment on your system. If your system environment is working then that is okay too. Do you have other UIs installed like AUTOMATIC1111 or ComfyUI?

actually I just got the error running from cmd so I take that back lol. I think it's more of a time between starting a fresh execution and it idling.

tailgunnerx commented 9 months ago

I noticed that I'm not getting this error if I run launch.py from cmd. What would be the difference vs. executing launch.py by clicking on it?

I think the difference would be that one is using the virtual environment venv and the other would be using the python environment on your system. If your system environment is working then that is okay too. Do you have other UIs installed like AUTOMATIC1111 or ComfyUI?

Yeah I have A1 for 1.5 but not running it atm.

machineminded commented 9 months ago

What operation within Fooocus causes the error from the OP?

tailgunnerx commented 9 months ago

What operation within Fooocus causes the error from the OP?

seems like it's all the functions that generate an image.

machineminded commented 9 months ago

After activating the venv with .\venv\Scripts\activate, and running pip show torch torchvision what is the output?

tailgunnerx commented 9 months ago

After activating the venv with .\venv\Scripts\activate, and running pip show torch torchvision what is the output?

(venv) C:\Fooocus-inswapper-main>pip show torch torchvision
Name: torch
Version: 2.2.0+cu118
Summary: Tensors and Dynamic neural networks in Python with strong GPU acceleration
Home-page: https://pytorch.org/
Author: PyTorch Team
Author-email: packages@pytorch.org
License: BSD-3
Location: c:\fooocus-inswapper-main\venv\lib\site-packages
Requires: filelock, fsspec, jinja2, networkx, sympy, typing-extensions
Required-by: accelerate, basicsr, lpips, peft, pytorch-lightning, timm, torchaudio, torchmetrics, torchsde, torchvision
---
Name: torchvision
Version: 0.17.0+cu118
Summary: image and video datasets and models for torch deep learning
Home-page: https://github.com/pytorch/vision
Author: PyTorch Core Team
Author-email: soumith@pytorch.org
License: BSD
Location: c:\fooocus-inswapper-main\venv\lib\site-packages
Requires: numpy, pillow, requests, torch
Required-by: basicsr, lpips, timm

machineminded commented 9 months ago

Can you post the full output from when the error occurs? Also, does it ever happen when starting, or only during image generation? I see a similar issue on the main repo here.

tailgunnerx commented 9 months ago

Errors seem to fluctuate, this is running inswapper on 0.

Microsoft Windows [Version 10.0.19045.3930]
(c) Microsoft Corporation. All rights reserved.

C:\Windows\System32>cd C:\Fooocus-inswapper-main

C:\Fooocus-inswapper-main>python launch.py
[System ARGV] ['launch.py']
You do not have [juggernautXL_v8Rundiffusion.safetensors] but you have [juggernautXL_version6Rundiffusion.safetensors].
Fooocus will use [juggernautXL_version6Rundiffusion.safetensors] to avoid downloading new models, but you are not using latest models.
Use --always-download-new-model to avoid fallback and always get new models.
Total VRAM 12282 MB, total RAM 32549 MB
Set vram state to: NORMAL_VRAM
Always offload VRAM
Device: cuda:0 NVIDIA GeForce RTX 4070 : native
VAE dtype: torch.bfloat16
Using pytorch cross attention
C:\Python310\lib\site-packages\diffusers\utils\outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
  torch.utils._pytree._register_pytree_node(
Refiner unloaded.
C:\Python310\lib\site-packages\diffusers\utils\outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
  torch.utils._pytree._register_pytree_node(
model_type EPS
UNet ADM Dimension 2816
Using pytorch attention in VAE
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
Using pytorch attention in VAE
extra {'cond_stage_model.clip_l.text_projection', 'cond_stage_model.clip_l.logit_scale'}
left over keys: dict_keys(['cond_stage_model.clip_l.transformer.text_model.embeddings.position_ids'])
Base model loaded: C:\Fooocus-inswapper-main\models\checkpoints\juggernautXL_version6Rundiffusion.safetensors
Request to load LoRAs [['sd_xl_offset_example-lora_1.0.safetensors', 0.1], ['None', 1.0], ['None', 1.0], ['None', 1.0], ['None', 1.0]] for model [C:\Fooocus-inswapper-main\models\checkpoints\juggernautXL_version6Rundiffusion.safetensors].
Loaded LoRA [C:\Fooocus-inswapper-main\models\loras\sd_xl_offset_example-lora_1.0.safetensors] for UNet [C:\Fooocus-inswapper-main\models\checkpoints\juggernautXL_version6Rundiffusion.safetensors] with 788 keys at weight 0.1.
Fooocus V2 Expansion: Vocab with 642 words.
Fooocus Expansion engine loaded for cuda:0, use_fp16 = True.
Requested to load SDXLClipModel
Requested to load GPT2LMHeadModel
Loading 2 new models
[Fooocus Model Management] Moving model(s) has taken 0.30 seconds
'Blocks' object has no attribute 'server_name'
Running on local URL:  http://127.0.0.1:7865

To create a public link, set `share=True` in `launch()`.
Inswapper: ENABLED
PhotoMaker: DISABLED
[Parameters] Adaptive CFG = 7
[Parameters] Sharpness = 2
[Parameters] ADM Scale = 1.5 : 0.8 : 0.3
[Parameters] CFG = 4.0
[Parameters] Seed = 4033774592796129292
[Parameters] Sampler = dpmpp_2m_sde_gpu - karras
[Parameters] Steps = 30 - 15
[Fooocus] Initializing ...
[Fooocus] Loading models ...
Refiner unloaded.
[Fooocus] Processing prompts ...
[Fooocus] Preparing Fooocus text #1 ...
[Prompt Expansion] 30 year old woman, cyberpunk, detailed deep bright pretty perfect cute sharp focus, beautiful intricate, elegant, highly inspired, extremely creative, color spread, very inspirational, innocent, light, magical, artistic, vibrant, symmetry, fine, hopeful, epic, cinematic, positive, pure, best, full, cheerful, thought, iconic, lovely, joyful, unique, awesome
[Fooocus] Preparing Fooocus text #2 ...
[Prompt Expansion] 30 year old woman, cyberpunk, intricate, elegant, highly detailed, extremely beautiful, glowing, sharp focus, symmetry, fine detail, inspired whole background, epic composition, colorful, polished, complex, amazing dynamic dramatic cinematic color, lovely colors, cute, wonderful atmosphere, professional, stunning, creative, positive, cheerful, unique, cool, best, winning, perfect
[Fooocus] Encoding positive #1 ...
[Fooocus Model Management] Moving model(s) has taken 0.12 seconds
[Fooocus] Encoding positive #2 ...
[Fooocus] Encoding negative #1 ...
[Fooocus] Encoding negative #2 ...
[Parameters] Denoising Strength = 1.0
[Parameters] Initial Latent shape: Image Space (896, 1152)
Preparation time: 1.74 seconds
[Sampler] refiner_swap_method = joint
[Sampler] sigma_min = 0.0291671771556139, sigma_max = 14.614643096923828
Requested to load SDXL
Loading 1 new model
[Fooocus Model Management] Moving model(s) has taken 1.44 seconds
  0%|                                                                                           | 0/30 [00:00<?, ?it/s]C:\Fooocus-inswapper-main\ldm_patched\ldm\modules\attention.py:325: UserWarning: 1Torch was not compiled with flash attention. (Triggered internally at ..\aten\src\ATen\native\transformers\cuda\sdp_utils.cpp:263.)
  out = torch.nn.functional.scaled_dot_product_attention(q, k, v, attn_mask=mask, dropout_p=0.0, is_causal=False)
100%|██████████████████████████████████████████████████████████████████████████████████| 30/30 [00:10<00:00,  2.78it/s]
Requested to load AutoencoderKL
Loading 1 new model
[Fooocus Model Management] Moving model(s) has taken 0.13 seconds
Inswapper: Target index: 0.0
Applied providers: ['AzureExecutionProvider', 'CPUExecutionProvider'], with options: {'AzureExecutionProvider': {}, 'CPUExecutionProvider': {}}
find model: ./checkpoints\models\buffalo_l\1k3d68.onnx landmark_3d_68 ['None', 3, 192, 192] 0.0 1.0
Applied providers: ['AzureExecutionProvider', 'CPUExecutionProvider'], with options: {'AzureExecutionProvider': {}, 'CPUExecutionProvider': {}}
find model: ./checkpoints\models\buffalo_l\2d106det.onnx landmark_2d_106 ['None', 3, 192, 192] 0.0 1.0
Applied providers: ['AzureExecutionProvider', 'CPUExecutionProvider'], with options: {'AzureExecutionProvider': {}, 'CPUExecutionProvider': {}}
find model: ./checkpoints\models\buffalo_l\det_10g.onnx detection [1, 3, '?', '?'] 127.5 128.0
Applied providers: ['AzureExecutionProvider', 'CPUExecutionProvider'], with options: {'AzureExecutionProvider': {}, 'CPUExecutionProvider': {}}
find model: ./checkpoints\models\buffalo_l\genderage.onnx genderage ['None', 3, 96, 96] 0.0 1.0
Applied providers: ['AzureExecutionProvider', 'CPUExecutionProvider'], with options: {'AzureExecutionProvider': {}, 'CPUExecutionProvider': {}}
find model: ./checkpoints\models\buffalo_l\w600k_r50.onnx recognition ['None', 3, 112, 112] 127.5 127.5
set det-size: (320, 320)
C:\Python310\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py:69: UserWarning: Specified provider 'CUDAExecutionProvider' is not in available provider names.Available providers: 'AzureExecutionProvider, CPUExecutionProvider'
  warnings.warn(
Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
inswapper-shape: [1, 3, 128, 128]
C:\Python310\lib\site-packages\insightface\utils\transform.py:68: FutureWarning: `rcond` parameter will change to the default of machine precision times ``max(M, N)`` where M and N are the input matrix dimensions.
To use the future default and silence this warning we advise to pass `rcond=None`, to keep using the old, explicitly pass `rcond=-1`.
  P = np.linalg.lstsq(X_homo, Y)[0].T # Affine matrix. 3 x 4
Replacing faces in target image from the left to the right by order
Traceback (most recent call last):
  File "C:\Fooocus-inswapper-main\modules\async_worker.py", line 860, in worker
    handler(task)
  File "C:\Python310\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Python310\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Fooocus-inswapper-main\modules\async_worker.py", line 812, in handler
    imgs = perform_face_swap(imgs, inswapper_source_image, inswapper_target_image_index)
  File "C:\Fooocus-inswapper-main\modules\face_swap.py", line 17, in perform_face_swap
    from inswapper.restoration import face_restoration,check_ckpts,set_realesrgan,torch,ARCH_REGISTRY,cv2
  File "C:\Fooocus-inswapper-main\inswapper\restoration.py", line 10, in <module>
    from basicsr.utils import imwrite, img2tensor, tensor2img
ModuleNotFoundError: No module named 'basicsr'
Total time: 17.66 seconds

machineminded commented 9 months ago

This is a different error. It says basicsr is not found. Please run

xcopy /E /I /Y inswapper\CodeFormer\CodeFormer\basicsr venv\Lib\site-packages\basicsr
xcopy /E /I /Y inswapper\CodeFormer\CodeFormer\facelib venv\Lib\site-packages\facelib

from the same directory where launch.py is located.

tailgunnerx commented 9 months ago

xcopy /E /I /Y inswapper\CodeFormer\CodeFormer\basicsr venv\Lib\site-packages\basicsr xcopy /E /I /Y inswapper\CodeFormer\CodeFormer\facelib venv\Lib\site-packages\facelib

just to be consistent, how many files are copying for you when you run these?

basicsr = 50 files copied facelib = 28 files copied

tailgunnerx commented 9 months ago

This is a different error. It says basicsr is not found. Please run
xcopy /E /I /Y inswapper\CodeFormer\CodeFormer\basicsr venv\Lib\site-packages\basicsr
xcopy /E /I /Y inswapper\CodeFormer\CodeFormer\facelib venv\Lib\site-packages\facelib
from the same directory where launch.py is located.

Microsoft Windows [Version 10.0.19045.3930]
(c) Microsoft Corporation. All rights reserved.

C:\Windows\System32>cd C:\Fooocus-inswapper-main

C:\Fooocus-inswapper-main>xcopy /E /I /Y inswapper\CodeFormer\CodeFormer\basicsr venv\Lib\site-packages\basicsr
inswapper\CodeFormer\CodeFormer\basicsr\setup.py
inswapper\CodeFormer\CodeFormer\basicsr\train.py
inswapper\CodeFormer\CodeFormer\basicsr\VERSION
inswapper\CodeFormer\CodeFormer\basicsr\version.py
inswapper\CodeFormer\CodeFormer\basicsr\__init__.py
inswapper\CodeFormer\CodeFormer\basicsr\archs\arcface_arch.py
inswapper\CodeFormer\CodeFormer\basicsr\archs\arch_util.py
inswapper\CodeFormer\CodeFormer\basicsr\archs\codeformer_arch.py
inswapper\CodeFormer\CodeFormer\basicsr\archs\rrdbnet_arch.py
inswapper\CodeFormer\CodeFormer\basicsr\archs\vgg_arch.py
inswapper\CodeFormer\CodeFormer\basicsr\archs\vqgan_arch.py
inswapper\CodeFormer\CodeFormer\basicsr\archs\__init__.py
inswapper\CodeFormer\CodeFormer\basicsr\data\data_sampler.py
inswapper\CodeFormer\CodeFormer\basicsr\data\data_util.py
inswapper\CodeFormer\CodeFormer\basicsr\data\prefetch_dataloader.py
inswapper\CodeFormer\CodeFormer\basicsr\data\transforms.py
inswapper\CodeFormer\CodeFormer\basicsr\data\__init__.py
inswapper\CodeFormer\CodeFormer\basicsr\losses\losses.py
inswapper\CodeFormer\CodeFormer\basicsr\losses\loss_util.py
inswapper\CodeFormer\CodeFormer\basicsr\losses\__init__.py
inswapper\CodeFormer\CodeFormer\basicsr\metrics\metric_util.py
inswapper\CodeFormer\CodeFormer\basicsr\metrics\psnr_ssim.py
inswapper\CodeFormer\CodeFormer\basicsr\metrics\__init__.py
inswapper\CodeFormer\CodeFormer\basicsr\models\__init__.py
inswapper\CodeFormer\CodeFormer\basicsr\ops\__init__.py
inswapper\CodeFormer\CodeFormer\basicsr\ops\dcn\deform_conv.py
inswapper\CodeFormer\CodeFormer\basicsr\ops\dcn\__init__.py
inswapper\CodeFormer\CodeFormer\basicsr\ops\dcn\src\deform_conv_cuda.cpp
inswapper\CodeFormer\CodeFormer\basicsr\ops\dcn\src\deform_conv_cuda_kernel.cu
inswapper\CodeFormer\CodeFormer\basicsr\ops\dcn\src\deform_conv_ext.cpp
inswapper\CodeFormer\CodeFormer\basicsr\ops\fused_act\fused_act.py
inswapper\CodeFormer\CodeFormer\basicsr\ops\fused_act\__init__.py
inswapper\CodeFormer\CodeFormer\basicsr\ops\fused_act\src\fused_bias_act.cpp
inswapper\CodeFormer\CodeFormer\basicsr\ops\fused_act\src\fused_bias_act_kernel.cu
inswapper\CodeFormer\CodeFormer\basicsr\ops\upfirdn2d\upfirdn2d.py
inswapper\CodeFormer\CodeFormer\basicsr\ops\upfirdn2d\__init__.py
inswapper\CodeFormer\CodeFormer\basicsr\ops\upfirdn2d\src\upfirdn2d.cpp
inswapper\CodeFormer\CodeFormer\basicsr\ops\upfirdn2d\src\upfirdn2d_kernel.cu
inswapper\CodeFormer\CodeFormer\basicsr\utils\dist_util.py
inswapper\CodeFormer\CodeFormer\basicsr\utils\download_util.py
inswapper\CodeFormer\CodeFormer\basicsr\utils\file_client.py
inswapper\CodeFormer\CodeFormer\basicsr\utils\img_util.py
inswapper\CodeFormer\CodeFormer\basicsr\utils\lmdb_util.py
inswapper\CodeFormer\CodeFormer\basicsr\utils\logger.py
inswapper\CodeFormer\CodeFormer\basicsr\utils\matlab_functions.py
inswapper\CodeFormer\CodeFormer\basicsr\utils\misc.py
inswapper\CodeFormer\CodeFormer\basicsr\utils\options.py
inswapper\CodeFormer\CodeFormer\basicsr\utils\realesrgan_utils.py
inswapper\CodeFormer\CodeFormer\basicsr\utils\registry.py
inswapper\CodeFormer\CodeFormer\basicsr\utils\__init__.py
50 File(s) copied

C:\Fooocus-inswapper-main>xcopy /E /I /Y inswapper\CodeFormer\CodeFormer\facelib venv\Lib\site-packages\facelib
inswapper\CodeFormer\CodeFormer\facelib\detection\align_trans.py
inswapper\CodeFormer\CodeFormer\facelib\detection\matlab_cp2tform.py
inswapper\CodeFormer\CodeFormer\facelib\detection\__init__.py
inswapper\CodeFormer\CodeFormer\facelib\detection\retinaface\retinaface.py
inswapper\CodeFormer\CodeFormer\facelib\detection\retinaface\retinaface_net.py
inswapper\CodeFormer\CodeFormer\facelib\detection\retinaface\retinaface_utils.py
inswapper\CodeFormer\CodeFormer\facelib\detection\yolov5face\face_detector.py
inswapper\CodeFormer\CodeFormer\facelib\detection\yolov5face\__init__.py
inswapper\CodeFormer\CodeFormer\facelib\detection\yolov5face\models\common.py
inswapper\CodeFormer\CodeFormer\facelib\detection\yolov5face\models\experimental.py
inswapper\CodeFormer\CodeFormer\facelib\detection\yolov5face\models\yolo.py
inswapper\CodeFormer\CodeFormer\facelib\detection\yolov5face\models\yolov5l.yaml
inswapper\CodeFormer\CodeFormer\facelib\detection\yolov5face\models\yolov5n.yaml
inswapper\CodeFormer\CodeFormer\facelib\detection\yolov5face\models\__init__.py
inswapper\CodeFormer\CodeFormer\facelib\detection\yolov5face\utils\autoanchor.py
inswapper\CodeFormer\CodeFormer\facelib\detection\yolov5face\utils\datasets.py
inswapper\CodeFormer\CodeFormer\facelib\detection\yolov5face\utils\extract_ckpt.py
inswapper\CodeFormer\CodeFormer\facelib\detection\yolov5face\utils\general.py
inswapper\CodeFormer\CodeFormer\facelib\detection\yolov5face\utils\torch_utils.py
inswapper\CodeFormer\CodeFormer\facelib\detection\yolov5face\utils\__init__.py
inswapper\CodeFormer\CodeFormer\facelib\parsing\bisenet.py
inswapper\CodeFormer\CodeFormer\facelib\parsing\parsenet.py
inswapper\CodeFormer\CodeFormer\facelib\parsing\resnet.py
inswapper\CodeFormer\CodeFormer\facelib\parsing\__init__.py
inswapper\CodeFormer\CodeFormer\facelib\utils\face_restoration_helper.py
inswapper\CodeFormer\CodeFormer\facelib\utils\face_utils.py
inswapper\CodeFormer\CodeFormer\facelib\utils\misc.py
inswapper\CodeFormer\CodeFormer\facelib\utils\__init__.py
28 File(s) copied

C:\Fooocus-inswapper-main>python launch.py
[System ARGV] ['launch.py']
You do not have [juggernautXL_v8Rundiffusion.safetensors] but you have [juggernautXL_version6Rundiffusion.safetensors].
Fooocus will use [juggernautXL_version6Rundiffusion.safetensors] to avoid downloading new models, but you are not using latest models.
Use --always-download-new-model to avoid fallback and always get new models.
Total VRAM 12282 MB, total RAM 32549 MB
Set vram state to: NORMAL_VRAM
Always offload VRAM
Device: cuda:0 NVIDIA GeForce RTX 4070 : native
VAE dtype: torch.bfloat16
Using pytorch cross attention
C:\Python310\lib\site-packages\diffusers\utils\outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
  torch.utils._pytree._register_pytree_node(
Refiner unloaded.
C:\Python310\lib\site-packages\diffusers\utils\outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
  torch.utils._pytree._register_pytree_node(
model_type EPS
UNet ADM Dimension 2816
Using pytorch attention in VAE
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
Using pytorch attention in VAE
extra {'cond_stage_model.clip_l.logit_scale', 'cond_stage_model.clip_l.text_projection'}
left over keys: dict_keys(['cond_stage_model.clip_l.transformer.text_model.embeddings.position_ids'])
Base model loaded: C:\Fooocus-inswapper-main\models\checkpoints\juggernautXL_version6Rundiffusion.safetensors
Request to load LoRAs [['sd_xl_offset_example-lora_1.0.safetensors', 0.1], ['None', 1.0], ['None', 1.0], ['None', 1.0], ['None', 1.0]] for model [C:\Fooocus-inswapper-main\models\checkpoints\juggernautXL_version6Rundiffusion.safetensors].
Loaded LoRA [C:\Fooocus-inswapper-main\models\loras\sd_xl_offset_example-lora_1.0.safetensors] for UNet [C:\Fooocus-inswapper-main\models\checkpoints\juggernautXL_version6Rundiffusion.safetensors] with 788 keys at weight 0.1.
Fooocus V2 Expansion: Vocab with 642 words.
Fooocus Expansion engine loaded for cuda:0, use_fp16 = True.
Requested to load SDXLClipModel
Requested to load GPT2LMHeadModel
Loading 2 new models
[Fooocus Model Management] Moving model(s) has taken 0.24 seconds
Running on local URL:  http://127.0.0.1:7865
App started successful. Use the app with http://127.0.0.1:7865/ or 127.0.0.1:7865

To create a public link, set `share=True` in `launch()`.
Inswapper: ENABLED
PhotoMaker: DISABLED
[Parameters] Adaptive CFG = 7
[Parameters] Sharpness = 2
[Parameters] ADM Scale = 1.5 : 0.8 : 0.3
[Parameters] CFG = 4.0
[Parameters] Seed = 3604631258951612005
[Parameters] Sampler = dpmpp_2m_sde_gpu - karras
[Parameters] Steps = 30 - 15
[Fooocus] Initializing ...
[Fooocus] Loading models ...
Refiner unloaded.
[Fooocus] Processing prompts ...
[Fooocus] Preparing Fooocus text #1 ...
[Prompt Expansion] 30 year old woman, cyberpunk game, gorgeous, amazing, elegant, intricate, highly detailed, dynamic light, great composition, atmosphere, fancy, cinematic, artistic, fine detail, pretty, perfect, colorful, sharp focus, winning, grand illumination, radiant, professional, best, creative, positive, thought, imagined, magic, dramatic, pure, coherent, cute
[Fooocus] Preparing Fooocus text #2 ...
[Prompt Expansion] 30 year old woman, cyberpunk game, cute, expressive detailed perfect royal elegant pretty, background light, ambient dynamic dramatic cinematic color, gorgeous atmosphere, sharp focus, fair composition, adventurous, lively, intricate, highly detail, beautiful, inspired, professional, winning, artistic, charismatic, thought best, inspirational, marvelous, fabulous, epic, singular, glorious, cool, vivid
[Fooocus] Encoding positive #1 ...
[Fooocus Model Management] Moving model(s) has taken 0.16 seconds
[Fooocus] Encoding positive #2 ...
[Fooocus] Encoding negative #1 ...
[Fooocus] Encoding negative #2 ...
[Parameters] Denoising Strength = 1.0
[Parameters] Initial Latent shape: Image Space (896, 1152)
Preparation time: 1.80 seconds
[Sampler] refiner_swap_method = joint
[Sampler] sigma_min = 0.0291671771556139, sigma_max = 14.614643096923828
Requested to load SDXL
Loading 1 new model
[Fooocus Model Management] Moving model(s) has taken 1.48 seconds
  0%|                                                                                           | 0/30 [00:00<?, ?it/s]C:\Fooocus-inswapper-main\ldm_patched\ldm\modules\attention.py:325: UserWarning: 1Torch was not compiled with flash attention. (Triggered internally at ..\aten\src\ATen\native\transformers\cuda\sdp_utils.cpp:263.)
  out = torch.nn.functional.scaled_dot_product_attention(q, k, v, attn_mask=mask, dropout_p=0.0, is_causal=False)
100%|██████████████████████████████████████████████████████████████████████████████████| 30/30 [00:10<00:00,  2.84it/s]
Requested to load AutoencoderKL
Loading 1 new model
[Fooocus Model Management] Moving model(s) has taken 0.10 seconds
Inswapper: Target index: 0.0
Applied providers: ['AzureExecutionProvider', 'CPUExecutionProvider'], with options: {'AzureExecutionProvider': {}, 'CPUExecutionProvider': {}}
find model: ./checkpoints\models\buffalo_l\1k3d68.onnx landmark_3d_68 ['None', 3, 192, 192] 0.0 1.0
Applied providers: ['AzureExecutionProvider', 'CPUExecutionProvider'], with options: {'AzureExecutionProvider': {}, 'CPUExecutionProvider': {}}
find model: ./checkpoints\models\buffalo_l\2d106det.onnx landmark_2d_106 ['None', 3, 192, 192] 0.0 1.0
Applied providers: ['AzureExecutionProvider', 'CPUExecutionProvider'], with options: {'AzureExecutionProvider': {}, 'CPUExecutionProvider': {}}
find model: ./checkpoints\models\buffalo_l\det_10g.onnx detection [1, 3, '?', '?'] 127.5 128.0
Applied providers: ['AzureExecutionProvider', 'CPUExecutionProvider'], with options: {'AzureExecutionProvider': {}, 'CPUExecutionProvider': {}}
find model: ./checkpoints\models\buffalo_l\genderage.onnx genderage ['None', 3, 96, 96] 0.0 1.0
Applied providers: ['AzureExecutionProvider', 'CPUExecutionProvider'], with options: {'AzureExecutionProvider': {}, 'CPUExecutionProvider': {}}
find model: ./checkpoints\models\buffalo_l\w600k_r50.onnx recognition ['None', 3, 112, 112] 127.5 127.5
set det-size: (320, 320)
C:\Python310\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py:69: UserWarning: Specified provider 'CUDAExecutionProvider' is not in available provider names.Available providers: 'AzureExecutionProvider, CPUExecutionProvider'
  warnings.warn(
Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
inswapper-shape: [1, 3, 128, 128]
C:\Python310\lib\site-packages\insightface\utils\transform.py:68: FutureWarning: `rcond` parameter will change to the default of machine precision times ``max(M, N)`` where M and N are the input matrix dimensions.
To use the future default and silence this warning we advise to pass `rcond=None`, to keep using the old, explicitly pass `rcond=-1`.
  P = np.linalg.lstsq(X_homo, Y)[0].T # Affine matrix. 3 x 4
Replacing faces in target image from the left to the right by order
Traceback (most recent call last):
  File "C:\Fooocus-inswapper-main\modules\async_worker.py", line 860, in worker
    handler(task)
  File "C:\Python310\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Python310\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Fooocus-inswapper-main\modules\async_worker.py", line 812, in handler
    imgs = perform_face_swap(imgs, inswapper_source_image, inswapper_target_image_index)
  File "C:\Fooocus-inswapper-main\modules\face_swap.py", line 17, in perform_face_swap
    from inswapper.restoration import face_restoration,check_ckpts,set_realesrgan,torch,ARCH_REGISTRY,cv2
  File "C:\Fooocus-inswapper-main\inswapper\restoration.py", line 10, in <module>
    from basicsr.utils import imwrite, img2tensor, tensor2img
ModuleNotFoundError: No module named 'basicsr'
Total time: 17.53 seconds

machineminded commented 9 months ago

It looks like you are running it outside of the venv. If you activate the venv and run what error do you get?

tailgunnerx commented 9 months ago

It looks like you are running it outside of the venv. If you activate the venv and run what error do you get?

C:\Windows\System32>cd C:\Fooocus-inswapper-main

C:\Fooocus-inswapper-main>.\venv\Scripts\activate

(venv) C:\Fooocus-inswapper-main>xcopy /E /I /Y inswapper\CodeFormer\CodeFormer\basicsr venv\Lib\site-packages\basicsr inswapper\CodeFormer\CodeFormer\basicsr\setup.py inswapper\CodeFormer\CodeFormer\basicsr\train.py inswapper\CodeFormer\CodeFormer\basicsr\VERSION inswapper\CodeFormer\CodeFormer\basicsr\version.py inswapper\CodeFormer\CodeFormer\basicsr__init.py inswapper\CodeFormer\CodeFormer\basicsr\archs\arcface_arch.py inswapper\CodeFormer\CodeFormer\basicsr\archs\arch_util.py inswapper\CodeFormer\CodeFormer\basicsr\archs\codeformer_arch.py inswapper\CodeFormer\CodeFormer\basicsr\archs\rrdbnet_arch.py inswapper\CodeFormer\CodeFormer\basicsr\archs\vgg_arch.py inswapper\CodeFormer\CodeFormer\basicsr\archs\vqgan_arch.py inswapper\CodeFormer\CodeFormer\basicsr\archs__init.py inswapper\CodeFormer\CodeFormer\basicsr\data\data_sampler.py inswapper\CodeFormer\CodeFormer\basicsr\data\data_util.py inswapper\CodeFormer\CodeFormer\basicsr\data\prefetch_dataloader.py inswapper\CodeFormer\CodeFormer\basicsr\data\transforms.py inswapper\CodeFormer\CodeFormer\basicsr\data__init__.py inswapper\CodeFormer\CodeFormer\basicsr\losses\losses.py inswapper\CodeFormer\CodeFormer\basicsr\losses\loss_util.py inswapper\CodeFormer\CodeFormer\basicsr\losses__init.py inswapper\CodeFormer\CodeFormer\basicsr\metrics\metric_util.py inswapper\CodeFormer\CodeFormer\basicsr\metrics\psnr_ssim.py inswapper\CodeFormer\CodeFormer\basicsr\metrics\init.py inswapper\CodeFormer\CodeFormer\basicsr\models\init.py inswapper\CodeFormer\CodeFormer\basicsr\ops\init__.py inswapper\CodeFormer\CodeFormer\basicsr\ops\dcn\deform_conv.py inswapper\CodeFormer\CodeFormer\basicsr\ops\dcn\init.py inswapper\CodeFormer\CodeFormer\basicsr\ops\dcn\src\deform_conv_cuda.cpp inswapper\CodeFormer\CodeFormer\basicsr\ops\dcn\src\deform_conv_cuda_kernel.cu inswapper\CodeFormer\CodeFormer\basicsr\ops\dcn\src\deform_conv_ext.cpp inswapper\CodeFormer\CodeFormer\basicsr\ops\fused_act\fused_act.py inswapper\CodeFormer\CodeFormer\basicsr\ops\fused_act\init__.py inswapper\CodeFormer\CodeFormer\basicsr\ops\fused_act\src\fused_bias_act.cpp inswapper\CodeFormer\CodeFormer\basicsr\ops\fused_act\src\fused_bias_act_kernel.cu inswapper\CodeFormer\CodeFormer\basicsr\ops\upfirdn2d\upfirdn2d.py inswapper\CodeFormer\CodeFormer\basicsr\ops\upfirdn2d\init.py inswapper\CodeFormer\CodeFormer\basicsr\ops\upfirdn2d\src\upfirdn2d.cpp inswapper\CodeFormer\CodeFormer\basicsr\ops\upfirdn2d\src\upfirdn2d_kernel.cu inswapper\CodeFormer\CodeFormer\basicsr\utils\dist_util.py inswapper\CodeFormer\CodeFormer\basicsr\utils\download_util.py inswapper\CodeFormer\CodeFormer\basicsr\utils\file_client.py inswapper\CodeFormer\CodeFormer\basicsr\utils\img_util.py inswapper\CodeFormer\CodeFormer\basicsr\utils\lmdb_util.py inswapper\CodeFormer\CodeFormer\basicsr\utils\logger.py inswapper\CodeFormer\CodeFormer\basicsr\utils\matlab_functions.py inswapper\CodeFormer\CodeFormer\basicsr\utils\misc.py inswapper\CodeFormer\CodeFormer\basicsr\utils\options.py inswapper\CodeFormer\CodeFormer\basicsr\utils\realesrgan_utils.py inswapper\CodeFormer\CodeFormer\basicsr\utils\registry.py inswapper\CodeFormer\CodeFormer\basicsr\utils\init__.py 50 File(s) copied

(venv) C:\Fooocus-inswapper-main>xcopy /E /I /Y inswapper\CodeFormer\CodeFormer\facelib venv\Lib\site-packages\facelib inswapper\CodeFormer\CodeFormer\facelib\detection\align_trans.py inswapper\CodeFormer\CodeFormer\facelib\detection\matlab_cp2tform.py inswapper\CodeFormer\CodeFormer\facelib\detection__init.py inswapper\CodeFormer\CodeFormer\facelib\detection\retinaface\retinaface.py inswapper\CodeFormer\CodeFormer\facelib\detection\retinaface\retinaface_net.py inswapper\CodeFormer\CodeFormer\facelib\detection\retinaface\retinaface_utils.py inswapper\CodeFormer\CodeFormer\facelib\detection\yolov5face\face_detector.py inswapper\CodeFormer\CodeFormer\facelib\detection\yolov5face__init.py inswapper\CodeFormer\CodeFormer\facelib\detection\yolov5face\models\common.py inswapper\CodeFormer\CodeFormer\facelib\detection\yolov5face\models\experimental.py inswapper\CodeFormer\CodeFormer\facelib\detection\yolov5face\models\yolo.py inswapper\CodeFormer\CodeFormer\facelib\detection\yolov5face\models\yolov5l.yaml inswapper\CodeFormer\CodeFormer\facelib\detection\yolov5face\models\yolov5n.yaml inswapper\CodeFormer\CodeFormer\facelib\detection\yolov5face\models\init.py inswapper\CodeFormer\CodeFormer\facelib\detection\yolov5face\utils\autoanchor.py inswapper\CodeFormer\CodeFormer\facelib\detection\yolov5face\utils\datasets.py inswapper\CodeFormer\CodeFormer\facelib\detection\yolov5face\utils\extract_ckpt.py inswapper\CodeFormer\CodeFormer\facelib\detection\yolov5face\utils\general.py inswapper\CodeFormer\CodeFormer\facelib\detection\yolov5face\utils\torch_utils.py inswapper\CodeFormer\CodeFormer\facelib\detection\yolov5face\utils\init.py inswapper\CodeFormer\CodeFormer\facelib\parsing\bisenet.py inswapper\CodeFormer\CodeFormer\facelib\parsing\parsenet.py inswapper\CodeFormer\CodeFormer\facelib\parsing\resnet.py inswapper\CodeFormer\CodeFormer\facelib\parsing\init__.py inswapper\CodeFormer\CodeFormer\facelib\utils\face_restoration_helper.py inswapper\CodeFormer\CodeFormer\facelib\utils\face_utils.py inswapper\CodeFormer\CodeFormer\facelib\utils\misc.py inswapper\CodeFormer\CodeFormer\facelib\utils\init__.py 28 File(s) copied

(venv) C:\Fooocus-inswapper-main>launch.py [System ARGV] ['C:\Fooocus-inswapper-main\launch.py'] You do not have [juggernautXL_v8Rundiffusion.safetensors] but you have [juggernautXL_version6Rundiffusion.safetensors]. Fooocus will use [juggernautXL_version6Rundiffusion.safetensors] to avoid downloading new models, but you are not using latest models. Use --always-download-new-model to avoid fallback and always get new models. Total VRAM 12282 MB, total RAM 32549 MB Set vram state to: NORMAL_VRAM Always offload VRAM Device: cuda:0 NVIDIA GeForce RTX 4070 : native VAE dtype: torch.bfloat16 Using pytorch cross attention C:\Fooocus-inswapper-main\venv\lib\site-packages\diffusers\utils\outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead. torch.utils._pytree._register_pytree_node( C:\Fooocus-inswapper-main\venv\lib\site-packages\diffusers\utils\outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead. torch.utils._pytree._register_pytree_node( C:\Fooocus-inswapper-main\venv\lib\site-packages\diffusers\utils\outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead. torch.utils._pytree._register_pytree_node( Refiner unloaded. model_type EPS UNet ADM Dimension 2816 Using pytorch attention in VAE Working with z of shape (1, 4, 32, 32) = 4096 dimensions. Using pytorch attention in VAE extra {'cond_stage_model.clip_l.text_projection', 'cond_stage_model.clip_l.logit_scale'} left over keys: dict_keys(['cond_stage_model.clip_l.transformer.text_model.embeddings.position_ids']) Base model loaded: C:\Fooocus-inswapper-main\models\checkpoints\juggernautXL_version6Rundiffusion.safetensors Request to load LoRAs [['sd_xl_offset_example-lora_1.0.safetensors', 0.1], ['None', 1.0], ['None', 1.0], ['None', 1.0], ['None', 1.0]] for model [C:\Fooocus-inswapper-main\models\checkpoints\juggernautXL_version6Rundiffusion.safetensors]. Loaded LoRA [C:\Fooocus-inswapper-main\models\loras\sd_xl_offset_example-lora_1.0.safetensors] for UNet [C:\Fooocus-inswapper-main\models\checkpoints\juggernautXL_version6Rundiffusion.safetensors] with 788 keys at weight 0.1. Fooocus V2 Expansion: Vocab with 642 words. Fooocus Expansion engine loaded for cuda:0, use_fp16 = True. Requested to load SDXLClipModel Requested to load GPT2LMHeadModel Loading 2 new models [Fooocus Model Management] Moving model(s) has taken 0.25 seconds Running on local URL: http://127.0.0.1:7865

To create a public link, set share=True in launch(). App started successful. Use the app with http://127.0.0.1:7865/ or 127.0.0.1:7865 Inswapper: ENABLED PhotoMaker: DISABLED [Parameters] Adaptive CFG = 7 [Parameters] Sharpness = 2 [Parameters] ADM Scale = 1.5 : 0.8 : 0.3 [Parameters] CFG = 4.0 [Parameters] Seed = 1078866180457242112 [Parameters] Sampler = dpmpp_2m_sde_gpu - karras [Parameters] Steps = 30 - 15 [Fooocus] Initializing ... [Fooocus] Loading models ... Refiner unloaded. [Fooocus] Processing prompts ... [Fooocus] Preparing Fooocus text #1 ... [Prompt Expansion] 30 year old woman, cyberpunk game, gorgeous, amazing, elegant, intricate, highly detailed, dynamic light, great composition, atmosphere, cinematic, sharp focus, winning, fine color, awesome, inspiring, creative, positive, vibrant, inspirational, beautiful, confident, pretty, attractive, deep background, professional, thought, best, unique, perfect, full, coherent [Fooocus] Preparing Fooocus text #2 ... [Prompt Expansion] 30 year old woman, cyberpunk game, gorgeous, amazing, elegant, intricate, highly detailed, dynamic light, great colors, beautiful, inspired, vibrant, deep aesthetic, very inspirational, glowing, attractive, original, full color, winning, fine detail, pretty, artistic, cool, creative, passionate, awesome, inspiring, illuminated, colorful, shiny, sharp, fantastic [Fooocus] Encoding positive #1 ... [Fooocus Model Management] Moving model(s) has taken 0.12 seconds [Fooocus] Encoding positive #2 ... [Fooocus] Encoding negative #1 ... [Fooocus] Encoding negative #2 ... [Parameters] Denoising Strength = 1.0 [Parameters] Initial Latent shape: Image Space (896, 1152) Preparation time: 2.07 seconds [Sampler] refiner_swap_method = joint [Sampler] sigma_min = 0.0291671771556139, sigma_max = 14.614643096923828 Requested to load SDXL Loading 1 new model [Fooocus Model Management] Moving model(s) has taken 1.41 seconds 0%| | 0/30 [00:00<?, ?it/s]C:\Fooocus-inswapper-main\ldm_patched\ldm\modules\attention.py:325: UserWarning: 1Torch was not compiled with flash attention. (Triggered internally at ..\aten\src\ATen\native\transformers\cuda\sdp_utils.cpp:263.) out = torch.nn.functional.scaled_dot_product_attention(q, k, v, attn_mask=mask, dropout_p=0.0, is_causal=False) 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 30/30 [00:10<00:00, 2.80it/s] Requested to load AutoencoderKL Loading 1 new model [Fooocus Model Management] Moving model(s) has taken 0.12 seconds Inswapper: Target index: 0.0 Applied providers: ['AzureExecutionProvider', 'CPUExecutionProvider'], with options: {'AzureExecutionProvider': {}, 'CPUExecutionProvider': {}} find model: ./checkpoints\models\buffalo_l\1k3d68.onnx landmark_3d_68 ['None', 3, 192, 192] 0.0 1.0 Applied providers: ['AzureExecutionProvider', 'CPUExecutionProvider'], with options: {'AzureExecutionProvider': {}, 'CPUExecutionProvider': {}} find model: ./checkpoints\models\buffalo_l\2d106det.onnx landmark_2d_106 ['None', 3, 192, 192] 0.0 1.0 Applied providers: ['AzureExecutionProvider', 'CPUExecutionProvider'], with options: {'AzureExecutionProvider': {}, 'CPUExecutionProvider': {}} find model: ./checkpoints\models\buffalo_l\det_10g.onnx detection [1, 3, '?', '?'] 127.5 128.0 Applied providers: ['AzureExecutionProvider', 'CPUExecutionProvider'], with options: {'AzureExecutionProvider': {}, 'CPUExecutionProvider': {}} find model: ./checkpoints\models\buffalo_l\genderage.onnx genderage ['None', 3, 96, 96] 0.0 1.0 Applied providers: ['AzureExecutionProvider', 'CPUExecutionProvider'], with options: {'AzureExecutionProvider': {}, 'CPUExecutionProvider': {}} find model: ./checkpoints\models\buffalo_l\w600k_r50.onnx recognition ['None', 3, 112, 112] 127.5 127.5 set det-size: (320, 320) C:\Fooocus-inswapper-main\venv\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py:69: UserWarning: Specified provider 'CUDAExecutionProvider' is not in available provider names.Available providers: 'AzureExecutionProvider, CPUExecutionProvider' warnings.warn( Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}} inswapper-shape: [1, 3, 128, 128] C:\Fooocus-inswapper-main\venv\lib\site-packages\insightface\utils\transform.py:68: FutureWarning: rcond parameter will change to the default of machine precision times max(M, N) where M and N are the input matrix dimensions. To use the future default and silence this warning we advise to pass rcond=None, to keep using the old, explicitly pass rcond=-1. P = np.linalg.lstsq(X_homo, Y)[0].T # Affine matrix. 3 x 4 Replacing faces in target image from the left to the right by order Traceback (most recent call last): File "C:\Fooocus-inswapper-main\modules\async_worker.py", line 860, in worker handler(task) File "C:\Fooocus-inswapper-main\venv\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context return func(*args, *kwargs) File "C:\Fooocus-inswapper-main\venv\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context return func(args, *kwargs) File "C:\Fooocus-inswapper-main\modules\async_worker.py", line 812, in handler imgs = perform_face_swap(imgs, inswapper_source_image, inswapper_target_image_index) File "C:\Fooocus-inswapper-main\modules\face_swap.py", line 17, in perform_face_swap from inswapper.restoration import face_restoration,check_ckpts,set_realesrgan,torch,ARCH_REGISTRY,cv2 File "C:\Fooocus-inswapper-main\inswapper\restoration.py", line 10, in from basicsr.utils import imwrite, img2tensor, tensor2img File "C:\Fooocus-inswapper-main\venv\lib\site-packages\basicsr__init__.py", line 3, in from .archs import File "C:\Fooocus-inswapper-main\venv\lib\site-packages\basicsr\archs__init.py", line 16, in _arch_modules = [importlib.import_module(f'basicsr.archs.{file_name}') for file_name in arch_filenames] File "C:\Fooocus-inswapper-main\venv\lib\site-packages\basicsr\archs\init.py", line 16, in _arch_modules = [importlib.import_module(f'basicsr.archs.{file_name}') for file_name in arch_filenames] File "C:\Python310\lib\importlib\init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "C:\Fooocus-inswapper-main\venv\lib\site-packages\basicsr\archs\discriminator_arch.py", line 90, in @ARCH_REGISTRY.register(suffix='basicsr') TypeError: Registry.register() got an unexpected keyword argument 'suffix' Total time: 18.25 seconds

machineminded commented 9 months ago

Something I noticed is that it is using CPUExecutionProvider during the inswapper routine. What video card do you have?

tailgunnerx commented 9 months ago

Something I noticed is that it is using CPUExecutionProvider during the inswapper routine. What video card do you have?

Gigabyte 4070

machineminded commented 9 months ago

With the venv activated, can you run a plain python command, and then type

import torch
torch.cuda.is_available()

tailgunnerx commented 9 months ago

(venv) C:\Fooocus-inswapper-main>python
Python 3.10.10 (tags/v3.10.10:aad5f6a, Feb  7 2023, 17:20:36) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> torch.cuda.is_available()
True
>>>

machineminded commented 9 months ago

I want to force it to use the GPU. For some reason it seems Fooocus can't use or see the GPU. Can you try the --always-gpu flag?

https://github.com/lllyasviel/Fooocus?tab=readme-ov-file#all-cmd-flags

python launch.py --always-gpu

tailgunnerx commented 9 months ago

I want to force it to use the GPU. For some reason it seems Fooocus can't use or see the GPU. Can you try the --always-gpu flag?

https://github.com/lllyasviel/Fooocus?tab=readme-ov-file#all-cmd-flags

python launch.py --always-gpu

running it now. Noticing the first generation is running super slow 7s per it.

I feel like Fooocus sees the gpu but the secondary script doesn't maybe? because the first generation is usually fast and working then it switches to a different module?

machineminded commented 9 months ago

If you check your task manager resources tab, can you see GPU memory being allocated, and GPU processor being used during the generation?

tailgunnerx commented 9 months ago

I want to force it to use the GPU. For some reason it seems Fooocus can't use or see the GPU. Can you try the --always-gpu flag?

https://github.com/lllyasviel/Fooocus?tab=readme-ov-file#all-cmd-flags

python launch.py --always-gpu

Microsoft Windows [Version 10.0.19045.3930]
(c) Microsoft Corporation. All rights reserved.

C:\Windows\System32>cd C:\Fooocus-inswapper-main

C:\Fooocus-inswapper-main>python launch.py --always-gpu
[System ARGV] ['launch.py', '--always-gpu']
You do not have [juggernautXL_v8Rundiffusion.safetensors] but you have [juggernautXL_version6Rundiffusion.safetensors].
Fooocus will use [juggernautXL_version6Rundiffusion.safetensors] to avoid downloading new models, but you are not using latest models.
Use --always-download-new-model to avoid fallback and always get new models.
Total VRAM 12282 MB, total RAM 32549 MB
Set vram state to: HIGH_VRAM
Always offload VRAM
Device: cuda:0 NVIDIA GeForce RTX 4070 : native
VAE dtype: torch.bfloat16
Using pytorch cross attention
C:\Python310\lib\site-packages\diffusers\utils\outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
  torch.utils._pytree._register_pytree_node(
Refiner unloaded.
C:\Python310\lib\site-packages\diffusers\utils\outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
  torch.utils._pytree._register_pytree_node(
model_type EPS
UNet ADM Dimension 2816
Using pytorch attention in VAE
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
Using pytorch attention in VAE
extra {'cond_stage_model.clip_l.text_projection', 'cond_stage_model.clip_l.logit_scale'}
left over keys: dict_keys(['cond_stage_model.clip_l.transformer.text_model.embeddings.position_ids'])
loaded straight to GPU
Requested to load SDXL
Loading 1 new model
[Fooocus Model Management] Moving model(s) has taken 0.12 seconds
Base model loaded: C:\Fooocus-inswapper-main\models\checkpoints\juggernautXL_version6Rundiffusion.safetensors
Request to load LoRAs [['sd_xl_offset_example-lora_1.0.safetensors', 0.1], ['None', 1.0], ['None', 1.0], ['None', 1.0], ['None', 1.0]] for model [C:\Fooocus-inswapper-main\models\checkpoints\juggernautXL_version6Rundiffusion.safetensors].
Loaded LoRA [C:\Fooocus-inswapper-main\models\loras\sd_xl_offset_example-lora_1.0.safetensors] for UNet [C:\Fooocus-inswapper-main\models\checkpoints\juggernautXL_version6Rundiffusion.safetensors] with 788 keys at weight 0.1.
Fooocus V2 Expansion: Vocab with 642 words.
Fooocus Expansion engine loaded for cuda:0, use_fp16 = True.
Requested to load SDXLClipModel
Requested to load GPT2LMHeadModel
Loading 2 new models
[Fooocus Model Management] Moving model(s) has taken 0.24 seconds
Running on local URL:  http://127.0.0.1:7865
App started successful. Use the app with http://127.0.0.1:7865/ or 127.0.0.1:7865

To create a public link, set `share=True` in `launch()`.
Inswapper: ENABLED
PhotoMaker: DISABLED
[Parameters] Adaptive CFG = 7
[Parameters] Sharpness = 2
[Parameters] ADM Scale = 1.5 : 0.8 : 0.3
[Parameters] CFG = 4.0
[Parameters] Seed = 984104948671436075
[Parameters] Sampler = dpmpp_2m_sde_gpu - karras
[Parameters] Steps = 30 - 15
[Fooocus] Initializing ...
[Fooocus] Loading models ...
Refiner unloaded.
[Fooocus] Processing prompts ...
[Fooocus] Preparing Fooocus text #1 ...
[Prompt Expansion] 30 year old woman, cyberpunk game, gorgeous, amazing, elegant, intricate, highly detailed, colorful, surreal, dramatic, sharp focus, beautiful light, epic, fine composition, complex, fantastic colors, very inspirational, wonderful atmosphere, professional, winning, perfect clear quality, hopeful, unique, artistic, positive, vibrant, attractive, cute, confident, pretty, glowing
[Fooocus] Preparing Fooocus text #2 ...
[Prompt Expansion] 30 year old woman, cyberpunk game, gorgeous, amazing, elegant, intricate, highly detailed, colorful, burning background, sharp focus, great composition, cinematic light, dynamic, appealing, shiny, beautiful, brilliant, singular, fantastic, creative, positive, hopeful, cute, determined, vibrant, inspirational, dramatic ambient, shining, illuminated, incredible, loving, attractive
[Fooocus] Encoding positive #1 ...
[Fooocus] Encoding positive #2 ...
[Fooocus] Encoding negative #1 ...
[Fooocus] Encoding negative #2 ...
[Parameters] Denoising Strength = 1.0
[Parameters] Initial Latent shape: Image Space (896, 1152)
Preparation time: 1.62 seconds
[Sampler] refiner_swap_method = joint
[Sampler] sigma_min = 0.0291671771556139, sigma_max = 14.614643096923828
Requested to load SDXL
Loading 1 new model
[Fooocus Model Management] Moving model(s) has taken 1.33 seconds
  0%|                                                                                           | 0/30 [00:00<?, ?it/s]C:\Fooocus-inswapper-main\ldm_patched\ldm\modules\attention.py:325: UserWarning: 1Torch was not compiled with flash attention. (Triggered internally at ..\aten\src\ATen\native\transformers\cuda\sdp_utils.cpp:263.)
  out = torch.nn.functional.scaled_dot_product_attention(q, k, v, attn_mask=mask, dropout_p=0.0, is_causal=False)
100%|██████████████████████████████████████████████████████████████████████████████████| 30/30 [03:29<00:00,  6.98s/it]
Requested to load AutoencoderKL
Loading 1 new model
[Fooocus Model Management] Moving model(s) has taken 0.16 seconds
Inswapper: Target index: 0.0
Applied providers: ['AzureExecutionProvider', 'CPUExecutionProvider'], with options: {'AzureExecutionProvider': {}, 'CPUExecutionProvider': {}}
find model: ./checkpoints\models\buffalo_l\1k3d68.onnx landmark_3d_68 ['None', 3, 192, 192] 0.0 1.0
Applied providers: ['AzureExecutionProvider', 'CPUExecutionProvider'], with options: {'AzureExecutionProvider': {}, 'CPUExecutionProvider': {}}
find model: ./checkpoints\models\buffalo_l\2d106det.onnx landmark_2d_106 ['None', 3, 192, 192] 0.0 1.0
Applied providers: ['AzureExecutionProvider', 'CPUExecutionProvider'], with options: {'AzureExecutionProvider': {}, 'CPUExecutionProvider': {}}
find model: ./checkpoints\models\buffalo_l\det_10g.onnx detection [1, 3, '?', '?'] 127.5 128.0
Applied providers: ['AzureExecutionProvider', 'CPUExecutionProvider'], with options: {'AzureExecutionProvider': {}, 'CPUExecutionProvider': {}}
find model: ./checkpoints\models\buffalo_l\genderage.onnx genderage ['None', 3, 96, 96] 0.0 1.0
Applied providers: ['AzureExecutionProvider', 'CPUExecutionProvider'], with options: {'AzureExecutionProvider': {}, 'CPUExecutionProvider': {}}
find model: ./checkpoints\models\buffalo_l\w600k_r50.onnx recognition ['None', 3, 112, 112] 127.5 127.5
set det-size: (320, 320)
C:\Python310\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py:69: UserWarning: Specified provider 'CUDAExecutionProvider' is not in available provider names.Available providers: 'AzureExecutionProvider, CPUExecutionProvider'
  warnings.warn(
Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
inswapper-shape: [1, 3, 128, 128]
C:\Python310\lib\site-packages\insightface\utils\transform.py:68: FutureWarning: `rcond` parameter will change to the default of machine precision times ``max(M, N)`` where M and N are the input matrix dimensions.
To use the future default and silence this warning we advise to pass `rcond=None`, to keep using the old, explicitly pass `rcond=-1`.
  P = np.linalg.lstsq(X_homo, Y)[0].T # Affine matrix. 3 x 4
Replacing faces in target image from the left to the right by order
Traceback (most recent call last):
  File "C:\Fooocus-inswapper-main\modules\async_worker.py", line 860, in worker
    handler(task)
  File "C:\Python310\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Python310\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Fooocus-inswapper-main\modules\async_worker.py", line 812, in handler
    imgs = perform_face_swap(imgs, inswapper_source_image, inswapper_target_image_index)
  File "C:\Fooocus-inswapper-main\modules\face_swap.py", line 17, in perform_face_swap
    from inswapper.restoration import face_restoration,check_ckpts,set_realesrgan,torch,ARCH_REGISTRY,cv2
  File "C:\Fooocus-inswapper-main\inswapper\restoration.py", line 10, in <module>
    from basicsr.utils import imwrite, img2tensor, tensor2img
ModuleNotFoundError: No module named 'basicsr'
Total time: 216.17 seconds

tailgunnerx commented 9 months ago

If you check your task manager resources tab, can you see GPU memory being allocated, and GPU processor being used during the generation?

It's deff hitting the GPU

machineminded commented 9 months ago

Be sure to activate the venv. The previous log shows venv is not activated.

machineminded commented 9 months ago

If you are using PhotoMaker, it requires at least 16GB VRAM, otherwise it will offload to system memory and generation time will be slow. That is a PhotoMaker requirement, nothing I can fix right now.

tailgunnerx commented 9 months ago

python launch.py --always-gpu

If you are using PhotoMaker, it requires at least 16GB VRAM, otherwise it will offload to system memory and generation time will be slow. That is a PhotoMaker requirement, nothing I can fix right now.

this is Inswapper

machineminded commented 9 months ago

I misspoke anyhow. PhotoMaker's docs say 11GB is required. I must have been thinking of InstantID which will be integrated soon.

When running inswapper, do you notice "CPUExecutionProvider" in the log? If so then that is probably why. Let me check the code to make sure it isn't defaulting to CPU for some reason. I am expecting to see "CUDAExecutionProvider".

tailgunnerx commented 9 months ago

Be sure to activate the venv. The previous log shows venv is not activated.

Microsoft Windows [Version 10.0.19045.3930]
(c) Microsoft Corporation. All rights reserved.

C:\Windows\System32>cd C:\Fooocus-inswapper-main

C:\Fooocus-inswapper-main>.\venv\Scripts\activate

(venv) C:\Fooocus-inswapper-main>python launch.py --always-gpu
[System ARGV] ['launch.py', '--always-gpu']
You do not have [juggernautXL_v8Rundiffusion.safetensors] but you have [juggernautXL_version6Rundiffusion.safetensors].
Fooocus will use [juggernautXL_version6Rundiffusion.safetensors] to avoid downloading new models, but you are not using latest models.
Use --always-download-new-model to avoid fallback and always get new models.
Total VRAM 12282 MB, total RAM 32549 MB
Set vram state to: HIGH_VRAM
Always offload VRAM
Device: cuda:0 NVIDIA GeForce RTX 4070 : native
VAE dtype: torch.bfloat16
Using pytorch cross attention
C:\Fooocus-inswapper-main\venv\lib\site-packages\diffusers\utils\outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
  torch.utils._pytree._register_pytree_node(
C:\Fooocus-inswapper-main\venv\lib\site-packages\diffusers\utils\outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
  torch.utils._pytree._register_pytree_node(
C:\Fooocus-inswapper-main\venv\lib\site-packages\diffusers\utils\outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
  torch.utils._pytree._register_pytree_node(
Refiner unloaded.
model_type EPS
UNet ADM Dimension 2816
Using pytorch attention in VAE
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
Using pytorch attention in VAE
extra {'cond_stage_model.clip_l.text_projection', 'cond_stage_model.clip_l.logit_scale'}
left over keys: dict_keys(['cond_stage_model.clip_l.transformer.text_model.embeddings.position_ids'])
loaded straight to GPU
Requested to load SDXL
Loading 1 new model
Base model loaded: C:\Fooocus-inswapper-main\models\checkpoints\juggernautXL_version6Rundiffusion.safetensors
Request to load LoRAs [['sd_xl_offset_example-lora_1.0.safetensors', 0.1], ['None', 1.0], ['None', 1.0], ['None', 1.0], ['None', 1.0]] for model [C:\Fooocus-inswapper-main\models\checkpoints\juggernautXL_version6Rundiffusion.safetensors].
Loaded LoRA [C:\Fooocus-inswapper-main\models\loras\sd_xl_offset_example-lora_1.0.safetensors] for UNet [C:\Fooocus-inswapper-main\models\checkpoints\juggernautXL_version6Rundiffusion.safetensors] with 788 keys at weight 0.1.
Fooocus V2 Expansion: Vocab with 642 words.
Fooocus Expansion engine loaded for cuda:0, use_fp16 = True.
Requested to load SDXLClipModel
Requested to load GPT2LMHeadModel
Loading 2 new models
[Fooocus Model Management] Moving model(s) has taken 0.25 seconds
'Blocks' object has no attribute 'server_name'
Running on local URL:  http://127.0.0.1:7865

To create a public link, set `share=True` in `launch()`.
Inswapper: ENABLED
PhotoMaker: DISABLED
[Parameters] Adaptive CFG = 7
[Parameters] Sharpness = 2
[Parameters] ADM Scale = 1.5 : 0.8 : 0.3
[Parameters] CFG = 4.0
[Parameters] Seed = 4294970823360950592
[Parameters] Sampler = dpmpp_2m_sde_gpu - karras
[Parameters] Steps = 30 - 15
[Fooocus] Initializing ...
[Fooocus] Loading models ...
Refiner unloaded.
[Fooocus] Processing prompts ...
[Fooocus] Preparing Fooocus text #1 ...
[Prompt Expansion] 30 year old woman, cyberpunk game, cute, expressive, strong, symmetry, intricate, detailed, background light, highly saturated, sharp focus, elegant, epic, polished, complex, amazing composition, professional, color, ambient dramatic incredible fine detail, stunning, creative, positive, attractive, confident, unique, best, pure, beautiful, colorful, illuminated, pretty
[Fooocus] Preparing Fooocus text #2 ...
[Prompt Expansion] 30 year old woman, cyberpunk game, gorgeous, detailed intricate, dramatic light, cinematic, highly saturated, sharp focus, extremely fine detail, striking, beautiful, elegant, perfect composition, professional, color, cute, great dynamic, atmosphere, creative, positive, quality, shiny, glowing, rich, hopeful, vibrant, passionate, pretty, determined, attractive, best
[Fooocus] Encoding positive #1 ...
[Fooocus] Encoding positive #2 ...
[Fooocus] Encoding negative #1 ...
[Fooocus] Encoding negative #2 ...
[Parameters] Denoising Strength = 1.0
[Parameters] Initial Latent shape: Image Space (896, 1152)
Preparation time: 1.59 seconds
[Sampler] refiner_swap_method = joint
[Sampler] sigma_min = 0.0291671771556139, sigma_max = 14.614643096923828
Requested to load SDXL
Loading 1 new model
[Fooocus Model Management] Moving model(s) has taken 1.22 seconds
  0%|                                                                                           | 0/30 [00:00<?, ?it/s]C:\Fooocus-inswapper-main\ldm_patched\ldm\modules\attention.py:325: UserWarning: 1Torch was not compiled with flash attention. (Triggered internally at ..\aten\src\ATen\native\transformers\cuda\sdp_utils.cpp:263.)
  out = torch.nn.functional.scaled_dot_product_attention(q, k, v, attn_mask=mask, dropout_p=0.0, is_causal=False)
100%|██████████████████████████████████████████████████████████████████████████████████| 30/30 [03:30<00:00,  7.02s/it]
Requested to load AutoencoderKL
Loading 1 new model
[Fooocus Model Management] Moving model(s) has taken 0.17 seconds
Inswapper: Target index: 0.0
Applied providers: ['AzureExecutionProvider', 'CPUExecutionProvider'], with options: {'AzureExecutionProvider': {}, 'CPUExecutionProvider': {}}
find model: ./checkpoints\models\buffalo_l\1k3d68.onnx landmark_3d_68 ['None', 3, 192, 192] 0.0 1.0
Applied providers: ['AzureExecutionProvider', 'CPUExecutionProvider'], with options: {'AzureExecutionProvider': {}, 'CPUExecutionProvider': {}}
find model: ./checkpoints\models\buffalo_l\2d106det.onnx landmark_2d_106 ['None', 3, 192, 192] 0.0 1.0
Applied providers: ['AzureExecutionProvider', 'CPUExecutionProvider'], with options: {'AzureExecutionProvider': {}, 'CPUExecutionProvider': {}}
find model: ./checkpoints\models\buffalo_l\det_10g.onnx detection [1, 3, '?', '?'] 127.5 128.0
Applied providers: ['AzureExecutionProvider', 'CPUExecutionProvider'], with options: {'AzureExecutionProvider': {}, 'CPUExecutionProvider': {}}
find model: ./checkpoints\models\buffalo_l\genderage.onnx genderage ['None', 3, 96, 96] 0.0 1.0
Applied providers: ['AzureExecutionProvider', 'CPUExecutionProvider'], with options: {'AzureExecutionProvider': {}, 'CPUExecutionProvider': {}}
find model: ./checkpoints\models\buffalo_l\w600k_r50.onnx recognition ['None', 3, 112, 112] 127.5 127.5
set det-size: (320, 320)
C:\Fooocus-inswapper-main\venv\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py:69: UserWarning: Specified provider 'CUDAExecutionProvider' is not in available provider names.Available providers: 'AzureExecutionProvider, CPUExecutionProvider'
  warnings.warn(
Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
inswapper-shape: [1, 3, 128, 128]
C:\Fooocus-inswapper-main\venv\lib\site-packages\insightface\utils\transform.py:68: FutureWarning: `rcond` parameter will change to the default of machine precision times ``max(M, N)`` where M and N are the input matrix dimensions.
To use the future default and silence this warning we advise to pass `rcond=None`, to keep using the old, explicitly pass `rcond=-1`.
  P = np.linalg.lstsq(X_homo, Y)[0].T # Affine matrix. 3 x 4
Replacing faces in target image from the left to the right by order
Traceback (most recent call last):
  File "C:\Fooocus-inswapper-main\modules\async_worker.py", line 860, in worker
    handler(task)
  File "C:\Fooocus-inswapper-main\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Fooocus-inswapper-main\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Fooocus-inswapper-main\modules\async_worker.py", line 812, in handler
    imgs = perform_face_swap(imgs, inswapper_source_image, inswapper_target_image_index)
  File "C:\Fooocus-inswapper-main\modules\face_swap.py", line 17, in perform_face_swap
    from inswapper.restoration import face_restoration,check_ckpts,set_realesrgan,torch,ARCH_REGISTRY,cv2
  File "C:\Fooocus-inswapper-main\inswapper\restoration.py", line 10, in <module>
    from basicsr.utils import imwrite, img2tensor, tensor2img
  File "C:\Fooocus-inswapper-main\venv\lib\site-packages\basicsr\__init__.py", line 3, in <module>
    from .archs import *
  File "C:\Fooocus-inswapper-main\venv\lib\site-packages\basicsr\archs\__init__.py", line 16, in <module>
    _arch_modules = [importlib.import_module(f'basicsr.archs.{file_name}') for file_name in arch_filenames]
  File "C:\Fooocus-inswapper-main\venv\lib\site-packages\basicsr\archs\__init__.py", line 16, in <listcomp>
    _arch_modules = [importlib.import_module(f'basicsr.archs.{file_name}') for file_name in arch_filenames]
  File "C:\Python310\lib\importlib\__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "C:\Fooocus-inswapper-main\venv\lib\site-packages\basicsr\archs\discriminator_arch.py", line 90, in <module>
    @ARCH_REGISTRY.register(suffix='basicsr')
TypeError: Registry.register() got an unexpected keyword argument 'suffix'
Total time: 217.28 seconds

machineminded commented 9 months ago

Can you check this link?

https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/9454#issuecomment-1575892313

Don't delete gfpgan. Lets try deleting the basicsr package from within your venv folder and copying it over using the xcopy command from earier:

xcopy /E /I /Y inswapper\CodeFormer\CodeFormer\basicsr venv\Lib\site-packages\basicsr

Then restart Fooocus. It still appears to be using CPU though. At this point the venv seems kind of hosed, and maybe we reclone to the repo or create a fresh venv.

tailgunnerx commented 9 months ago

Can you check this link?

AUTOMATIC1111/stable-diffusion-webui#9454 (comment)

Don't delete gfpgan. Lets try deleting the basicsr package from within your venv folder and copying it over using the xcopy command from earier:

xcopy /E /I /Y inswapper\CodeFormer\CodeFormer\basicsr venv\Lib\site-packages\basicsr

Then restart Fooocus. It still appears to be using CPU though. At this point the venv seems kind of hosed, and maybe we reclone to the repo or create a fresh venv.

Ok so it did work but I forgot to use the GPU only flag (not sure it matters but notable) Going to also try this with the GPU flag

Microsoft Windows [Version 10.0.19045.3930]
(c) Microsoft Corporation. All rights reserved.

C:\Windows\System32>cd C:\Fooocus-inswapper-main

C:\Fooocus-inswapper-main>.\venv\Scripts\activate

(venv) C:\Fooocus-inswapper-main>xcopy /E /I /Y inswapper\CodeFormer\CodeFormer\basicsr venv\Lib\site-packages\basicsr
inswapper\CodeFormer\CodeFormer\basicsr\setup.py
inswapper\CodeFormer\CodeFormer\basicsr\train.py
inswapper\CodeFormer\CodeFormer\basicsr\VERSION
inswapper\CodeFormer\CodeFormer\basicsr\version.py
inswapper\CodeFormer\CodeFormer\basicsr\__init__.py
inswapper\CodeFormer\CodeFormer\basicsr\archs\arcface_arch.py
inswapper\CodeFormer\CodeFormer\basicsr\archs\arch_util.py
inswapper\CodeFormer\CodeFormer\basicsr\archs\codeformer_arch.py
inswapper\CodeFormer\CodeFormer\basicsr\archs\rrdbnet_arch.py
inswapper\CodeFormer\CodeFormer\basicsr\archs\vgg_arch.py
inswapper\CodeFormer\CodeFormer\basicsr\archs\vqgan_arch.py
inswapper\CodeFormer\CodeFormer\basicsr\archs\__init__.py
inswapper\CodeFormer\CodeFormer\basicsr\data\data_sampler.py
inswapper\CodeFormer\CodeFormer\basicsr\data\data_util.py
inswapper\CodeFormer\CodeFormer\basicsr\data\prefetch_dataloader.py
inswapper\CodeFormer\CodeFormer\basicsr\data\transforms.py
inswapper\CodeFormer\CodeFormer\basicsr\data\__init__.py
inswapper\CodeFormer\CodeFormer\basicsr\losses\losses.py
inswapper\CodeFormer\CodeFormer\basicsr\losses\loss_util.py
inswapper\CodeFormer\CodeFormer\basicsr\losses\__init__.py
inswapper\CodeFormer\CodeFormer\basicsr\metrics\metric_util.py
inswapper\CodeFormer\CodeFormer\basicsr\metrics\psnr_ssim.py
inswapper\CodeFormer\CodeFormer\basicsr\metrics\__init__.py
inswapper\CodeFormer\CodeFormer\basicsr\models\__init__.py
inswapper\CodeFormer\CodeFormer\basicsr\ops\__init__.py
inswapper\CodeFormer\CodeFormer\basicsr\ops\dcn\deform_conv.py
inswapper\CodeFormer\CodeFormer\basicsr\ops\dcn\__init__.py
inswapper\CodeFormer\CodeFormer\basicsr\ops\dcn\src\deform_conv_cuda.cpp
inswapper\CodeFormer\CodeFormer\basicsr\ops\dcn\src\deform_conv_cuda_kernel.cu
inswapper\CodeFormer\CodeFormer\basicsr\ops\dcn\src\deform_conv_ext.cpp
inswapper\CodeFormer\CodeFormer\basicsr\ops\fused_act\fused_act.py
inswapper\CodeFormer\CodeFormer\basicsr\ops\fused_act\__init__.py
inswapper\CodeFormer\CodeFormer\basicsr\ops\fused_act\src\fused_bias_act.cpp
inswapper\CodeFormer\CodeFormer\basicsr\ops\fused_act\src\fused_bias_act_kernel.cu
inswapper\CodeFormer\CodeFormer\basicsr\ops\upfirdn2d\upfirdn2d.py
inswapper\CodeFormer\CodeFormer\basicsr\ops\upfirdn2d\__init__.py
inswapper\CodeFormer\CodeFormer\basicsr\ops\upfirdn2d\src\upfirdn2d.cpp
inswapper\CodeFormer\CodeFormer\basicsr\ops\upfirdn2d\src\upfirdn2d_kernel.cu
inswapper\CodeFormer\CodeFormer\basicsr\utils\dist_util.py
inswapper\CodeFormer\CodeFormer\basicsr\utils\download_util.py
inswapper\CodeFormer\CodeFormer\basicsr\utils\file_client.py
inswapper\CodeFormer\CodeFormer\basicsr\utils\img_util.py
inswapper\CodeFormer\CodeFormer\basicsr\utils\lmdb_util.py
inswapper\CodeFormer\CodeFormer\basicsr\utils\logger.py
inswapper\CodeFormer\CodeFormer\basicsr\utils\matlab_functions.py
inswapper\CodeFormer\CodeFormer\basicsr\utils\misc.py
inswapper\CodeFormer\CodeFormer\basicsr\utils\options.py
inswapper\CodeFormer\CodeFormer\basicsr\utils\realesrgan_utils.py
inswapper\CodeFormer\CodeFormer\basicsr\utils\registry.py
inswapper\CodeFormer\CodeFormer\basicsr\utils\__init__.py
50 File(s) copied

(venv) C:\Fooocus-inswapper-main>python launch.py
[System ARGV] ['launch.py']
You do not have [juggernautXL_v8Rundiffusion.safetensors] but you have [juggernautXL_version6Rundiffusion.safetensors].
Fooocus will use [juggernautXL_version6Rundiffusion.safetensors] to avoid downloading new models, but you are not using latest models.
Use --always-download-new-model to avoid fallback and always get new models.
Total VRAM 12282 MB, total RAM 32549 MB
Set vram state to: NORMAL_VRAM
Always offload VRAM
Device: cuda:0 NVIDIA GeForce RTX 4070 : native
VAE dtype: torch.bfloat16
Using pytorch cross attention
C:\Fooocus-inswapper-main\venv\lib\site-packages\diffusers\utils\outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
  torch.utils._pytree._register_pytree_node(
C:\Fooocus-inswapper-main\venv\lib\site-packages\diffusers\utils\outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
  torch.utils._pytree._register_pytree_node(
C:\Fooocus-inswapper-main\venv\lib\site-packages\diffusers\utils\outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
  torch.utils._pytree._register_pytree_node(
Refiner unloaded.
model_type EPS
UNet ADM Dimension 2816
Using pytorch attention in VAE
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
Using pytorch attention in VAE
extra {'cond_stage_model.clip_l.text_projection', 'cond_stage_model.clip_l.logit_scale'}
left over keys: dict_keys(['cond_stage_model.clip_l.transformer.text_model.embeddings.position_ids'])
Base model loaded: C:\Fooocus-inswapper-main\models\checkpoints\juggernautXL_version6Rundiffusion.safetensors
Request to load LoRAs [['sd_xl_offset_example-lora_1.0.safetensors', 0.1], ['None', 1.0], ['None', 1.0], ['None', 1.0], ['None', 1.0]] for model [C:\Fooocus-inswapper-main\models\checkpoints\juggernautXL_version6Rundiffusion.safetensors].
Loaded LoRA [C:\Fooocus-inswapper-main\models\loras\sd_xl_offset_example-lora_1.0.safetensors] for UNet [C:\Fooocus-inswapper-main\models\checkpoints\juggernautXL_version6Rundiffusion.safetensors] with 788 keys at weight 0.1.
Fooocus V2 Expansion: Vocab with 642 words.
Fooocus Expansion engine loaded for cuda:0, use_fp16 = True.
Requested to load SDXLClipModel
Requested to load GPT2LMHeadModel
Loading 2 new models
[Fooocus Model Management] Moving model(s) has taken 0.24 seconds
Running on local URL:  http://127.0.0.1:7865

To create a public link, set `share=True` in `launch()`.
App started successful. Use the app with http://127.0.0.1:7865/ or 127.0.0.1:7865
Inswapper: ENABLED
PhotoMaker: DISABLED
[Parameters] Adaptive CFG = 7
[Parameters] Sharpness = 2
[Parameters] ADM Scale = 1.5 : 0.8 : 0.3
[Parameters] CFG = 4.0
[Parameters] Seed = 3868999066546115190
[Parameters] Sampler = dpmpp_2m_sde_gpu - karras
[Parameters] Steps = 30 - 15
[Fooocus] Initializing ...
[Fooocus] Loading models ...
Refiner unloaded.
[Fooocus] Processing prompts ...
[Fooocus] Preparing Fooocus text #1 ...
[Prompt Expansion] 30 year old woman, cyberpunk game, gorgeous, amazing, elegant, intricate, highly detailed, dynamic light, great composition, atmosphere, cinematic, thought, perfect clear focus, inspired colors, open background, professional, artistic, positive, healing, beautiful, glowing, unique, cute, fine detail, very inspirational, creative, vibrant, lovely, best, hopeful, pure
[Fooocus] Preparing Fooocus text #2 ...
[Prompt Expansion] 30 year old woman, cyberpunk game, gorgeous, amazing, elegant, intricate, highly detailed, dynamic light, great composition, atmosphere, fancy, cinematic, artistic, fine detail, perfect colorful, beautiful, calm, relaxed, pretty, very inspirational, vibrant color, complex, brilliant, thought, futuristic, epic, hopeful, dramatic ambient, deep colors, coherent, symmetry
[Fooocus] Encoding positive #1 ...
[Fooocus Model Management] Moving model(s) has taken 0.12 seconds
[Fooocus] Encoding positive #2 ...
[Fooocus] Encoding negative #1 ...
[Fooocus] Encoding negative #2 ...
[Parameters] Denoising Strength = 1.0
[Parameters] Initial Latent shape: Image Space (896, 1152)
Preparation time: 1.79 seconds
[Sampler] refiner_swap_method = joint
[Sampler] sigma_min = 0.0291671771556139, sigma_max = 14.614643096923828
Requested to load SDXL
Loading 1 new model
[Fooocus Model Management] Moving model(s) has taken 1.43 seconds
  0%|                                                                                           | 0/30 [00:00<?, ?it/s]C:\Fooocus-inswapper-main\ldm_patched\ldm\modules\attention.py:325: UserWarning: 1Torch was not compiled with flash attention. (Triggered internally at ..\aten\src\ATen\native\transformers\cuda\sdp_utils.cpp:263.)
  out = torch.nn.functional.scaled_dot_product_attention(q, k, v, attn_mask=mask, dropout_p=0.0, is_causal=False)
100%|██████████████████████████████████████████████████████████████████████████████████| 30/30 [00:10<00:00,  2.75it/s]
Requested to load AutoencoderKL
Loading 1 new model
[Fooocus Model Management] Moving model(s) has taken 0.11 seconds
Inswapper: Target index: 0.0
Applied providers: ['AzureExecutionProvider', 'CPUExecutionProvider'], with options: {'AzureExecutionProvider': {}, 'CPUExecutionProvider': {}}
find model: ./checkpoints\models\buffalo_l\1k3d68.onnx landmark_3d_68 ['None', 3, 192, 192] 0.0 1.0
Applied providers: ['AzureExecutionProvider', 'CPUExecutionProvider'], with options: {'AzureExecutionProvider': {}, 'CPUExecutionProvider': {}}
find model: ./checkpoints\models\buffalo_l\2d106det.onnx landmark_2d_106 ['None', 3, 192, 192] 0.0 1.0
Applied providers: ['AzureExecutionProvider', 'CPUExecutionProvider'], with options: {'AzureExecutionProvider': {}, 'CPUExecutionProvider': {}}
find model: ./checkpoints\models\buffalo_l\det_10g.onnx detection [1, 3, '?', '?'] 127.5 128.0
Applied providers: ['AzureExecutionProvider', 'CPUExecutionProvider'], with options: {'AzureExecutionProvider': {}, 'CPUExecutionProvider': {}}
find model: ./checkpoints\models\buffalo_l\genderage.onnx genderage ['None', 3, 96, 96] 0.0 1.0
Applied providers: ['AzureExecutionProvider', 'CPUExecutionProvider'], with options: {'AzureExecutionProvider': {}, 'CPUExecutionProvider': {}}
find model: ./checkpoints\models\buffalo_l\w600k_r50.onnx recognition ['None', 3, 112, 112] 127.5 127.5
set det-size: (320, 320)
C:\Fooocus-inswapper-main\venv\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py:69: UserWarning: Specified provider 'CUDAExecutionProvider' is not in available provider names.Available providers: 'AzureExecutionProvider, CPUExecutionProvider'
  warnings.warn(
Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
inswapper-shape: [1, 3, 128, 128]
C:\Fooocus-inswapper-main\venv\lib\site-packages\insightface\utils\transform.py:68: FutureWarning: `rcond` parameter will change to the default of machine precision times ``max(M, N)`` where M and N are the input matrix dimensions.
To use the future default and silence this warning we advise to pass `rcond=None`, to keep using the old, explicitly pass `rcond=-1`.
  P = np.linalg.lstsq(X_homo, Y)[0].T # Affine matrix. 3 x 4
Replacing faces in target image from the left to the right by order
Image generated with private log at: C:\Fooocus-inswapper-main\outputs\2024-02-08\log.html
Generating and saving time: 20.39 seconds
[Sampler] refiner_swap_method = joint
[Sampler] sigma_min = 0.0291671771556139, sigma_max = 14.614643096923828
Requested to load SDXL
Loading 1 new model
[Fooocus Model Management] Moving model(s) has taken 0.78 seconds
100%|██████████████████████████████████████████████████████████████████████████████████| 30/30 [00:10<00:00,  2.81it/s]
Requested to load AutoencoderKL
Loading 1 new model
[Fooocus Model Management] Moving model(s) has taken 0.11 seconds
Inswapper: Target index: 0.0
Applied providers: ['AzureExecutionProvider', 'CPUExecutionProvider'], with options: {'AzureExecutionProvider': {}, 'CPUExecutionProvider': {}}
find model: ./checkpoints\models\buffalo_l\1k3d68.onnx landmark_3d_68 ['None', 3, 192, 192] 0.0 1.0
Applied providers: ['AzureExecutionProvider', 'CPUExecutionProvider'], with options: {'AzureExecutionProvider': {}, 'CPUExecutionProvider': {}}
find model: ./checkpoints\models\buffalo_l\2d106det.onnx landmark_2d_106 ['None', 3, 192, 192] 0.0 1.0
Applied providers: ['AzureExecutionProvider', 'CPUExecutionProvider'], with options: {'AzureExecutionProvider': {}, 'CPUExecutionProvider': {}}
find model: ./checkpoints\models\buffalo_l\det_10g.onnx detection [1, 3, '?', '?'] 127.5 128.0
Applied providers: ['AzureExecutionProvider', 'CPUExecutionProvider'], with options: {'AzureExecutionProvider': {}, 'CPUExecutionProvider': {}}
find model: ./checkpoints\models\buffalo_l\genderage.onnx genderage ['None', 3, 96, 96] 0.0 1.0
Applied providers: ['AzureExecutionProvider', 'CPUExecutionProvider'], with options: {'AzureExecutionProvider': {}, 'CPUExecutionProvider': {}}
find model: ./checkpoints\models\buffalo_l\w600k_r50.onnx recognition ['None', 3, 112, 112] 127.5 127.5
set det-size: (320, 320)
Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
inswapper-shape: [1, 3, 128, 128]
C:\Fooocus-inswapper-main\venv\lib\site-packages\insightface\utils\transform.py:68: FutureWarning: `rcond` parameter will change to the default of machine precision times ``max(M, N)`` where M and N are the input matrix dimensions.
To use the future default and silence this warning we advise to pass `rcond=None`, to keep using the old, explicitly pass `rcond=-1`.
  P = np.linalg.lstsq(X_homo, Y)[0].T # Affine matrix. 3 x 4
Replacing faces in target image from the left to the right by order
Image generated with private log at: C:\Fooocus-inswapper-main\outputs\2024-02-08\log.html
Generating and saving time: 18.07 seconds
Requested to load SDXLClipModel
Requested to load GPT2LMHeadModel
Loading 2 new models
[Fooocus Model Management] Moving model(s) has taken 0.30 seconds
Total time: 43.88 seconds

tailgunnerx commented 9 months ago

When using the GPU flag it's really slow (but seems like it works for both the initial image generation and the face swap aspect at least).

tailgunnerx commented 9 months ago

Going to test some more and see if xcopy /E /I /Y inswapper\CodeFormer\CodeFormer\basicsr venv\Lib\site-packages\basicsr fixes this issue.

machineminded commented 9 months ago

For some reason inswapper is using only CPU according to your log above. Based on the inswapper code in the repo, it will use CPU if cuda is not available. So I would need to dig into why that could be happening. I will share my log output shortly.

tailgunnerx commented 9 months ago

For some reason inswapper is using only CPU according to your log above. Based on the inswapper code in the repo, it will use CPU if cuda is not available. So I would need to dig into why that could be happening. I will share my log output shortly.

Maybe it's best optimized for CPU? Because the GPU only flag it's really slow.

tailgunnerx commented 9 months ago

So I ran a few generations on PhotoMaker then went back to Inswapper and then got this again:

  File "C:\Fooocus-inswapper-main\venv\lib\site-packages\transformers\models\gpt2\modeling_gpt2.py", line 202, in _attn
    attn_weights = torch.where(causal_mask, attn_weights.to(attn_weights.dtype), mask_value)
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!
Total time: 0.92 seconds

machineminded commented 9 months ago

Do normal Fooocus features work okay? Is it only doing this on inswapper now?

machineminded commented 9 months ago

PhotoMaker will load a lot of stuff into VRAM. I think in your case we should try disabling PhotoMaker if switching to inswapper. Using both on the 4070 may not work due to RAM limitations so some tensors are probably loaded on different devices which is causing the error. Mostly a speculation right now though.

Unchecking the "Enabled" will unload PhotoMaker from memory. I may need to update this so it only runs when the PhotoMaker tab is active, and navigating away from the PhotoMaker tab will also unload the model. On 24GB card you can run PhotoMaker and inswapper at the same time.

The current behavior is that if inswapper is enabled, it will always execute regardless of what you are doing. PhotoMaker will only execute if the tab is active.

tailgunnerx commented 9 months ago

Do normal Fooocus features work okay? Is it only doing this on inswapper now?

I will test it out some more and let you know

tailgunnerx commented 9 months ago

@machineminded Seems like most of everything works until you start switching back and forth between the different tabs (PhotoMaker deff seems to be the culprit). It seems like one is maybe reserving the CUDA then brining out the CPU when you switch or vice-versa and the error starts appearing.

What actually is the difference between Inswapper and PhotoMaker?

machineminded commented 9 months ago

I do notice on my local that inswapper is also using CPUExecutionProvider. I believe I know the root cause, and will test soon.

Inswapper uses insightface to swap faces from a source image to a target image. Meaning, it will overlay a face onto an existing image.

PhotoMaker and InstantID will both use input images to guide the latent space towards a result. They also use huggingface diffusers, whereas the default Fooocus pipelines use ksampler.

If you want to disable PhotoMaker, I would uncheck "Enabled" to unload it from VRAM.

tailgunnerx commented 9 months ago

I do notice on my local that inswapper is also using CPUExecutionProvider. I believe I know the root cause, and will test soon.

Inswapper uses insightface to swap faces from a source image to a target image. Meaning, it will overlay a face onto an existing image.

PhotoMaker and InstantID will both use input images to guide the latent space towards a result. They also use huggingface diffusers, whereas the default Fooocus pipelines use ksampler.

If you want to disable PhotoMaker, I would uncheck "Enabled" to unload it from VRAM.

It doesn't seem to be an issue as long as I don't check it in the first place. But yeah I do notice some pixilation around faces when using Inswapper, so that wouldn't happen on PhotoMaker then I suppose?

machineminded commented 9 months ago

Inswapper and PhotoMaker are two separate ways to re-create someone's identity. Since PhotoMaker isn't overlaying an image you wouldn't see the pixelation, but the likeness may not be perfect either.

inswapper PhotoMaker

tailgunnerx commented 9 months ago

Gotcha, it's still pretty damn impressive though considering you're not actually training a whole model.

machineminded commented 9 months ago

You should be able to resolve the inswapper CPU issue by installing onnxruntime-gpu:

pip install onnxruntime-gpu

tailgunnerx commented 9 months ago

Does this version have Clip Skip? I noticed some models require it

tailgunnerx commented 9 months ago

You should be able to resolve the inswapper CPU issue by installing onnxruntime-gpu:

pip install onnxruntime-gpu

Installing collected packages: onnxruntime-gpu
ERROR: Could not install packages due to an OSError: [WinError 5] Access is denied: 'C:\\Fooocus-inswapper-main\\venv\\Lib\\site-packages\\onnxruntime\\capi\\onnxruntime_providers_shared.dll'
Check the permissions.

machineminded commented 9 months ago

Gotcha, it's still pretty damn impressive though considering you're not actually training a whole model.

I should be integrating InstantID soon, which is even more impressive.

Does this version have Clip Skip? I noticed some models require it

PhotoMaker and InstantID would support clip skip, but Fooocus doesn't provide anywhere to set and pass this value. Even the base Fooocus doesn't allow a way to set clip skip.

You should be able to resolve the inswapper CPU issue by installing onnxruntime-gpu: pip install onnxruntime-gpu

Installing collected packages: onnxruntime-gpu
ERROR: Could not install packages due to an OSError: [WinError 5] Access is denied: 'C:\\Fooocus-inswapper-main\\venv\\Lib\\site-packages\\onnxruntime\\capi\\onnxruntime_providers_shared.dll'
Check the permissions.

If the app is running, maybe try stopping the app and run the command again.

machineminded / Fooocus-inswapper

"RuntimeError: Found at least two devices" #10