Closed tndgpp closed 4 weeks ago
Hey there, @tndgpp! I'm here to help you out with any bugs, questions, or contributions you have. Let me know how I can assist you!
The crash during the processing of the first enabled tab when running ADetailer on AUTOMATIC1111 version v1.10.1 is likely due to compatibility issues. ADetailer does not support WebUI versions below 1.6.0, and there are specific imports and checks for compatibility with versions up to 1.9.0. If there are significant changes or updates in version v1.10.1 that are not compatible with ADetailer, it could cause the extension to crash [1][2].
Additionally, the args.py
file for ADetailer defines a comprehensive list of arguments and configurations that the extension uses. If any of these configurations are not properly handled or if there are discrepancies in how these arguments are processed in version v1.10.1, it could lead to crashes [3].
To address this issue, you may need to review and update the handling of specific arguments and configurations to ensure compatibility with version v1.10.1.
To continue talking to Dosu, mention @dosu.
@dosu After some testing, I realized that the error is triggered if mediapipe_face_mesh and mediapipe_face_mesh_eyes_only are among the enabled models. In this case, adetailer crashes as soon as it tries to use one of these models, otherwise everything runs smoothly. I don't know if this is relevant in general but this is what happens in my case.
To resolve the crash with ADetailer on AUTOMATIC1111 version v1.10.1 when the mediapipe_face_mesh
and mediapipe_face_mesh_eyes_only
templates are enabled, ensure that the mediapipe
library is correctly imported and used in these functions. Here is the relevant part of the code that handles these templates:
def mediapipe_face_mesh(
image: Image.Image, confidence: float = 0.3
) -> PredictOutput[int]:
import mediapipe as mp
mp_face_mesh = mp.solutions.face_mesh
draw_util = mp.solutions.drawing_utils
drawing_styles = mp.solutions.drawing_styles
w, h = image.size
with mp_face_mesh.FaceMesh(
static_image_mode=True, max_num_faces=20, min_detection_confidence=confidence
) as face_mesh:
arr = np.array(image)
pred = face_mesh.process(arr)
if pred.multi_face_landmarks is None:
return PredictOutput()
preview = arr.copy()
masks = []
for landmarks in pred.multi_face_landmarks:
draw_util.draw_landmarks(
image=preview,
landmark_list=landmarks,
connections=mp_face_mesh.FACEMESH_TESSELATION,
landmark_drawing_spec=None,
connection_drawing_spec=drawing_styles.get_default_face_mesh_tesselation_style(),
)
points = np.array(
[[land.x * w, land.y * h] for land in landmarks.landmark], dtype=int
)
outline = cv2.convexHull(points).reshape(-1).tolist()
mask = Image.new("L", image.size, "black")
draw = ImageDraw.Draw(mask)
draw.polygon(outline, fill="white")
masks.append(mask)
bboxes = create_bbox_from_mask(masks, image.size)
preview = Image.fromarray(preview)
return PredictOutput(bboxes=bboxes, masks=masks, preview=preview)
def mediapipe_face_mesh_eyes_only(
image: Image.Image, confidence: float = 0.3
) -> PredictOutput[int]:
import mediapipe as mp
mp_face_mesh = mp.solutions.face_mesh
left_idx = np.array(list(mp_face_mesh.FACEMESH_LEFT_EYE)).flatten()
right_idx = np.array(list(mp_face_mesh.FACEMESH_RIGHT_EYE)).flatten()
w, h = image.size
with mp_face_mesh.FaceMesh(
static_image_mode=True, max_num_faces=20, min_detection_confidence=confidence
) as face_mesh:
arr = np.array(image)
pred = face_mesh.process(arr)
if pred.multi_face_landmarks is None:
return PredictOutput()
preview = image.copy()
masks = []
for landmarks in pred.multi_face_landmarks:
points = np.array(
[[land.x * w, land.y * h] for land in landmarks.landmark], dtype=int
)
left_eyes = points[left_idx]
right_eyes = points[right_idx]
left_outline = cv2.convexHull(left_eyes).reshape(-1).tolist()
right_outline = cv2.convexHull(right_eyes).reshape(-1).tolist()
mask = Image.new("L", image.size, "black")
draw = ImageDraw.Draw(mask)
for outline in (left_outline, right_outline):
draw.polygon(outline, fill="white")
masks.append(mask)
bboxes = create_bbox_from_mask(masks, image.size)
preview = draw_preview(preview, bboxes, masks)
return PredictOutput(bboxes=bboxes, masks=masks, preview=preview)
Ensure that the mediapipe
library is installed in your environment:
pip install mediapipe
If the crash persists, it might be due to other issues not covered in the provided context [1].
To continue talking to Dosu, mention @dosu.
@dosu mediapipe
library is actually installed (version 0.10.14)
$ pip show mediapipe
Name: mediapipe
Version: 0.10.14
Summary: MediaPipe is the simplest way for researchers and developers to build world-class ML solutions and applications for mobile, edge, cloud and the web.
Home-page: https://github.com/google/mediapipe
Author: The MediaPipe Authors
Author-email: mediapipe@google.com
License: Apache 2.0
Location: /opt/AUTOMATIC1111/stable-diffusion-webui/venv/lib/python3.11/site-packages
Requires: absl-py, attrs, flatbuffers, jax, jaxlib, matplotlib, numpy, opencv-contrib-python, protobuf, sounddevice
Required-by: handrefinerportable
With version 24.8.0, adetailer does not crash anymore when encountering those mediapipe models, it simply does not detect anything and proceeds with the next tab enabled (if any). So, this specific bug can be considered solved. However, the author should clarify whether those mediapipe models should be considered deprecated or not.
Describe the bug
Running AUTOMATIC1111 version: [v1.10.1] • python: 3.11.2 • torch: 2.1.2+cu121 • xformers: 0.0.23.post1 • gradio: 3.41.2 ADetailer searches the element defined in the model of the first enabled tab, finds the element, refines it but then crashes, failing to inpaint the main image
Steps to reproduce
I wrote a simple prompt ("beautiful girl standing in a city street, intricate detail") with no negative, enabled three ADetailer tabs (face_yolov8n.pt, mediapipe_face_mesh_eyes_only, hand_yolov8n.pt in this order), and started the image generation. The image was generated, but ADetailer crashed while processing first tab.
Screenshots
No response
Console logs, from start to end.
List of installed extensions
ADetailer