search

Bing-su / adetailer

Auto detecting, masking and inpainting with detection model.
GNU Affero General Public License v3.0
4.07k stars 315 forks source link

[Bug]: expected mat1 and mat2 to have the same dtype, but got: float != struct c10::Half #568

Closed INTstinkt closed 4 months ago

INTstinkt commented 5 months ago

Describe the bug

If i try to use ADetailer in A1111 i get this error

Steps to reproduce

Activate Adetailer, generate picture

Screenshots

No response

Console logs, from start to end.

*** Error running postprocess_image: C:\AI\stable-diffusion-webui\extensions\adetailer\scripts\!adetailer.py
    Traceback (most recent call last):
      File "C:\AI\stable-diffusion-webui\modules\scripts.py", line 856, in postprocess_image
        script.postprocess_image(p, pp, *script_args)
      File "C:\AI\stable-diffusion-webui\extensions\adetailer\adetailer\traceback.py", line 159, in wrapper
        raise error from None
    RuntimeError:
    ╭──────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
    │                                                   System info                                                    │
    │ ┏━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ │
    │ ┃             ┃ Value                                                                                          ┃ │
    │ ┡━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩ │
    │ │    Platform │ Windows-10-10.0.19045-SP0                                                                      │ │
    │ │      Python │ 3.10.11 (tags/v3.10.11:7d4cc5a, Apr  5 2023, 00:38:17) [MSC v.1929 64 bit (AMD64)]             │ │
    │ │     Version │ v1.8.0                                                                                         │ │
    │ │      Commit │ bef51aed032c0aaa5cfd80445bc4cf0d85b408b5                                                       │ │
    │ │ Commandline │ ['launch.py', '--xformers', '--theme', 'dark']                                                 │ │
    │ │   Libraries │ {'torch': '2.1.2+cu121', 'torchvision': '0.16.2+cu121', 'ultralytics': '8.1.29', 'mediapipe':  │ │
    │ │             │ '0.10.9'}                                                                                      │ │
    │ └─────────────┴────────────────────────────────────────────────────────────────────────────────────────────────┘ │
    │                                                      Inputs                                                      │
    │ ┏━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ │
    │ ┃                 ┃ Value                                                                                      ┃ │
    │ ┡━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩ │
    │ │          prompt │ D&D style portrait, fantasy, digital art, Female Naga, Short bob hair, highly detailed,    │ │
    │ │                 │ creative, feminine, close-up, concept art, detailed face, intense, rich vivid colors,      │ │
    │ │                 │ ambient background, luxurious, fine polished, cool colors, illustrious, aesthetic, dynamic │ │
    │ │                 │ dramatic beautiful full taking, surreal                                                    │ │
    │ │ negative_prompt │ (photograph:1.3), realism, (photorealism:1.3), modern, ordinary, mundane, simplistic,      │ │
    │ │                 │ minimalism, plain, simple, full body shot, masculine, character sheet, worst quality, low  │ │
    │ │                 │ quality, normal quality, lowres, low details, oversaturated, undersaturated, overexposed,  │ │
    │ │                 │ underexposed, grayscale, bw, bad photo, bad photography, bad art, watermark, signature,    │ │
    │ │                 │ text font, username, error, logo, words, letters, digits, autograph, trademark, name,      │ │
    │ │                 │ blur, blurry, grainy, ugly, asymmetrical, poorly lit, bad shadow, draft, cropped, out of   │ │
    │ │                 │ frame, cut off, censored, jpeg artifacts, out of focus, glitch, duplicate,                 │ │
    │ │          n_iter │ 1                                                                                          │ │
    │ │      batch_size │ 1                                                                                          │ │
    │ │           width │ 1024                                                                                       │ │
    │ │          height │ 1024                                                                                       │ │
    │ │    sampler_name │ DPM++ SDE Karras                                                                           │ │
    │ │       enable_hr │ False                                                                                      │ │
    │ │     hr_upscaler │ Latent                                                                                     │ │
    │ │      checkpoint │ XL\animagineXLV31_v31.safetensors                                                          │ │
    │ │             vae │ Automatic                                                                                  │ │
    │ │            unet │ None                                                                                       │ │
    │ └─────────────────┴────────────────────────────────────────────────────────────────────────────────────────────┘ │
    │                 ADetailer                                                                                        │
    │ ┏━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓                                                                        │
    │ ┃                     ┃ Value           ┃                                                                        │
    │ ┡━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩                                                                        │
    │ │             version │ 24.3.5          │                                                                        │
    │ │            ad_model │ face_yolov8s.pt │                                                                        │
    │ │           ad_prompt │                 │                                                                        │
    │ │  ad_negative_prompt │                 │                                                                        │
    │ │ ad_controlnet_model │ None            │                                                                        │
    │ │              is_api │ False           │                                                                        │
    │ └─────────────────────┴─────────────────┘                                                                        │
    │ ╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮             │
    │ │ C:\AI\stable-diffusion-webui\extensions\adetailer\adetailer\traceback.py:139 in wrapper          │             │
    │ │                                                                                                  │             │
    │ │   138 │   │   try:                                                                               │             │
    │ │ ❱ 139 │   │   │   return func(*args, **kwargs)                                                   │             │
    │ │   140 │   │   except Exception as e:                                                             │             │
    │ │                                                                                                  │             │
    │ │ C:\AI\stable-diffusion-webui\extensions\adetailer\scripts\!adetailer.py:824 in postprocess_image │             │
    │ │                                                                                                  │             │
    │ │    823 │   │   │   │   │   continue                                                              │             │
    │ │ ❱  824 │   │   │   │   is_processed |= self._postprocess_image_inner(p, pp, args, n=n)           │             │
    │ │    825                                                                                           │             │
    │ │                                                                                                  │             │
    │ │ C:\AI\stable-diffusion-webui\extensions\adetailer\scripts\!adetailer.py:747 in                   │             │
    │ │ _postprocess_image_inner                                                                         │             │
    │ │                                                                                                  │             │
    │ │    746 │   │   with change_torch_load():                                                         │             │
    │ │ ❱  747 │   │   │   pred = predictor(ad_model, pp.image, args.ad_confidence, **kwargs)            │             │
    │ │    748                                                                                           │             │
    │ │                                                                                                  │             │
    │ │ C:\AI\stable-diffusion-webui\extensions\adetailer\adetailer\ultralytics.py:29 in                 │             │
    │ │ ultralytics_predict                                                                              │             │
    │ │                                                                                                  │             │
    │ │   28 │   apply_classes(model, model_path, classes)                                               │             │
    │ │ ❱ 29 │   pred = model(image, conf=confidence, device=device)                                     │             │
    │ │   30                                                                                             │             │
    │ │                                                                                                  │             │
    │ │ C:\AI\stable-diffusion-webui\venv\lib\site-packages\ultralytics\engine\model.py:169 in __call__  │             │
    │ │                                                                                                  │             │
    │ │   168 │   │   """                                                                                │             │
    │ │ ❱ 169 │   │   return self.predict(source, stream, **kwargs)                                      │             │
    │ │   170                                                                                            │             │
    │ │                                                                                                  │             │
    │ │ C:\AI\stable-diffusion-webui\venv\lib\site-packages\ultralytics\engine\model.py:432 in predict   │             │
    │ │                                                                                                  │             │
    │ │   431 │   │   │   self.predictor = predictor or self._smart_load("predictor")(overrides=args,    │             │
    │ │ ❱ 432 │   │   │   self.predictor.setup_model(model=self.model, verbose=is_cli)                   │             │
    │ │   433 │   │   else:  # only update args if predictor is already setup                            │             │
    │ │                                                                                                  │             │
    │ │ C:\AI\stable-diffusion-webui\venv\lib\site-packages\ultralytics\engine\predictor.py:297 in       │             │
    │ │ setup_model                                                                                      │             │
    │ │                                                                                                  │             │
    │ │   296 │   │   """Initialize YOLO model with given parameters and set it to evaluation mode."""   │             │
    │ │ ❱ 297 │   │   self.model = AutoBackend(                                                          │             │
    │ │   298 │   │   │   weights=model or self.args.model,                                              │             │
    │ │                                                                                                  │             │
    │ │ C:\AI\stable-diffusion-webui\venv\lib\site-packages\torch\utils\_contextlib.py:115 in            │             │
    │ │ decorate_context                                                                                 │             │
    │ │                                                                                                  │             │
    │ │   114 │   │   with ctx_factory():                                                                │             │
    │ │ ❱ 115 │   │   │   return func(*args, **kwargs)                                                   │             │
    │ │   116                                                                                            │             │
    │ │                                                                                                  │             │
    │ │ C:\AI\stable-diffusion-webui\venv\lib\site-packages\ultralytics\nn\autobackend.py:143 in         │             │
    │ │ __init__                                                                                         │             │
    │ │                                                                                                  │             │
    │ │   142 │   │   │   model = weights.to(device)                                                     │             │
    │ │ ❱ 143 │   │   │   model = model.fuse(verbose=verbose) if fuse else model                         │             │
    │ │   144 │   │   │   if hasattr(model, "kpt_shape"):                                                │             │
    │ │                                                                                                  │             │
    │ │ C:\AI\stable-diffusion-webui\venv\lib\site-packages\ultralytics\nn\tasks.py:184 in fuse          │             │
    │ │                                                                                                  │             │
    │ │    183 │   │   │   │   │   │   m.fuse_convs()                                                    │             │
    │ │ ❱  184 │   │   │   │   │   m.conv = fuse_conv_and_bn(m.conv, m.bn)  # update conv                │             │
    │ │    185 │   │   │   │   │   delattr(m, "bn")  # remove batchnorm                                  │             │
    │ │                                                                                                  │             │
    │ │ C:\AI\stable-diffusion-webui\venv\lib\site-packages\ultralytics\utils\torch_utils.py:195 in      │             │
    │ │ fuse_conv_and_bn                                                                                 │             │
    │ │                                                                                                  │             │
    │ │   194 │   b_bn = bn.bias - bn.weight.mul(bn.running_mean).div(torch.sqrt(bn.running_var + bn.e   │             │
    │ │ ❱ 195 │   fusedconv.bias.copy_(torch.mm(w_bn, b_conv.reshape(-1, 1)).reshape(-1) + b_bn)         │             │
    │ │   196                                                                                            │             │
    │ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯             │
    │ RuntimeError: expected mat1 and mat2 to have the same dtype, but got: float != struct c10::Half

List of installed extensions

No response

dosubot[bot] commented 5 months ago

Hey @INTstinkt! Great to see you again around here. Running into a bit of a snag, I see? Let's take a closer look together.

Thank you for providing detailed information about the issue you're encountering. The error message you're seeing, "expected mat1 and mat2 to have the same dtype, but got: float != struct c10::Half", typically occurs when there's a mismatch in the data types of tensors being operated on in PyTorch.

Given that the error is occurring within the !adetailer.py script, it's likely that the mismatch is happening during a matrix multiplication or a similar operation within the model or processing pipeline. However, without the specific lines of code where the error occurs, it's difficult to pinpoint the exact cause.

Here are a few suggestions to resolve this issue:

  1. Ensure Consistent Tensor Data Types: Before performing operations on tensors, ensure they are of the same data type. You can convert a tensor to a different data type using the .to() method. For example, if you have a tensor a of dtype float32 and want to convert it to float16, you can use a.to(torch.float16).

  2. Check Model and Input Data Types: Verify that the model and all input tensors are using compatible data types. If the model is expected to work with float16 tensors for efficiency, ensure that all inputs are converted to float16 before passing them to the model.

  3. Automatic Mixed Precision (AMP): If you are using PyTorch and performing operations that could benefit from mixed precision (using both float16 and float32 for different parts of the computation to balance speed and accuracy), consider using PyTorch's Automatic Mixed Precision (AMP) to handle data type conversions automatically. This can be particularly useful when working with models that are sensitive to data types.

Unfortunately, I wasn't able to find the specific function or method in the !adetailer.py script where the error is being thrown, nor any known issues with the torch or torchvision versions used that could cause this data type mismatch error.

If you're comfortable with it, you could try debugging the !adetailer.py script to find the exact location where the error occurs. This would involve running the script in a Python debugger and stepping through the code until the error is thrown. Once you've found the location, you can check the data types of the tensors being operated on and make the necessary adjustments.

If you need further assistance, please provide more information about the specific lines of code where the error occurs. This will help in providing a more accurate solution.

Sources

#### About Dosu This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research. Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant. If you want to continue the conversation, start your reply with @dosu-bot.
Bing-su commented 5 months ago

At least, just generating an image doesn't cause this error. You don't give the full console logs, nor do you give any information about the extensions you installed, so there's nothing I can investigate.