Make `union_inpaint` preprocessor

[light-and-ray]

          > Inpaint: Does not work right now

I can get something. Just use "None" preprocessor + image where black pixels mean mask. This comfy workflow do the same

You can get masking workflow from metadata

But there is still a black contour, it happens because: Resize Mode Just Resize Crop and Resize Resize and Fill

Originally posted by @light-and-ray in https://github.com/Mikubill/sd-webui-controlnet/issues/2998#issuecomment-2284186360

[light-and-ray]

So the best way I think is to create a union_inpaint preprocessor, which pastes black pixels under mask after image resized

[light-and-ray]

I've given up 😔

class PreprocessorInpaintUnion(Preprocessor):
    def __init__(self):
        super().__init__(name="inpaint_union")
        self.tags = ["Inpaint"]
        self.slider_resolution = PreprocessorParameter(visible=False)
        self.sorting_priority = 50
        self.accepts_mask = True
        self.requires_mask = True

    def __call__(
        self,
        input_image,
        resolution,
        slider_1=None,
        slider_2=None,
        slider_3=None,
        **kwargs
    ):
        input_image[:, :, :3][input_image[:, :, 3]!= 0] = 0
        input_image = input_image[:, :, :3]

        return Preprocessor.Result(
            value=input_image,
            display_images=input_image[None, :, :, :],
        )

This preprocessor gives a good image which is included as display_images, but the processed image is a total garbage

If I pass preprocessed image into None preprocessor, it works, but with a black contour

class PreprocessorInpaintUnion(Preprocessor):
    def __init__(self):
        super().__init__(name="inpaint_union")
        self.tags = ["Inpaint"]
        self.slider_resolution = PreprocessorParameter(visible=False)
        self.sorting_priority = 50
        self.accepts_mask = True
        self.requires_mask = True

    def __call__(
        self,
        input_image,
        resolution,
        slider_1=None,
        slider_2=None,
        slider_3=None,
        **kwargs
    ):
        # input_image[:, :, :3][input_image[:, :, 3]!= 0] = 0
        # input_image = input_image[:, :, :3]

        image = Image.fromarray(
                np.ascontiguousarray(input_image[:, :, :3].clip(0, 255).astype(np.uint8)).copy()
            )
        mask = Image.fromarray(
                np.ascontiguousarray(input_image[:, :, 3].clip(0, 255).astype(np.uint8)).copy()
            )
        black_image = Image.new('RGB', image.size, (0, 0, 0))
        image.paste(black_image, mask=mask)
        masked_image = HWC3(np.asarray(image).astype(np.uint8))

        return Preprocessor.Result(
            value=masked_image,
            display_images=masked_image[None, :, :, :],
        )

In this code I tried to make the same in pillow, and the result is better. But still worse then I would pass it into None preprocessor. And also there is a contour, although I've even set pixel perfect and resolution in call is 1024p

I have no idea how to work with numpy images even with llama as assistant 😔