Open light-and-ray opened 2 months ago
So the best way I think is to create a union_inpaint
preprocessor, which pastes black pixels under mask after image resized
I've given up 😔
class PreprocessorInpaintUnion(Preprocessor):
def __init__(self):
super().__init__(name="inpaint_union")
self.tags = ["Inpaint"]
self.slider_resolution = PreprocessorParameter(visible=False)
self.sorting_priority = 50
self.accepts_mask = True
self.requires_mask = True
def __call__(
self,
input_image,
resolution,
slider_1=None,
slider_2=None,
slider_3=None,
**kwargs
):
input_image[:, :, :3][input_image[:, :, 3]!= 0] = 0
input_image = input_image[:, :, :3]
return Preprocessor.Result(
value=input_image,
display_images=input_image[None, :, :, :],
)
This preprocessor gives a good image which is included as display_images, but the processed image is a total garbage
If I pass preprocessed image into None preprocessor, it works, but with a black contour
class PreprocessorInpaintUnion(Preprocessor):
def __init__(self):
super().__init__(name="inpaint_union")
self.tags = ["Inpaint"]
self.slider_resolution = PreprocessorParameter(visible=False)
self.sorting_priority = 50
self.accepts_mask = True
self.requires_mask = True
def __call__(
self,
input_image,
resolution,
slider_1=None,
slider_2=None,
slider_3=None,
**kwargs
):
# input_image[:, :, :3][input_image[:, :, 3]!= 0] = 0
# input_image = input_image[:, :, :3]
image = Image.fromarray(
np.ascontiguousarray(input_image[:, :, :3].clip(0, 255).astype(np.uint8)).copy()
)
mask = Image.fromarray(
np.ascontiguousarray(input_image[:, :, 3].clip(0, 255).astype(np.uint8)).copy()
)
black_image = Image.new('RGB', image.size, (0, 0, 0))
image.paste(black_image, mask=mask)
masked_image = HWC3(np.asarray(image).astype(np.uint8))
return Preprocessor.Result(
value=masked_image,
display_images=masked_image[None, :, :, :],
)
In this code I tried to make the same in pillow, and the result is better. But still worse then I would pass it into None preprocessor. And also there is a contour, although I've even set pixel perfect and resolution in call is 1024p
I have no idea how to work with numpy images even with llama as assistant 😔
I can get something. Just use "None" preprocessor + image where black pixels mean mask. This comfy workflow do the same
You can get masking workflow from metadata
But there is still a black contour, it happens because: Resize Mode Just Resize Crop and Resize Resize and Fill
Originally posted by @light-and-ray in https://github.com/Mikubill/sd-webui-controlnet/issues/2998#issuecomment-2284186360