Feature Request: Empty Mask on Image Upload Without Running SAM

[Sj-Si]

Problem

While preprocessing images, I find that some small text cannot be easily detected using segmentation. Thus I find it easier to just manually mask out parts of the image that I want to inpaint. However the current state of this module requires that I first upload my image, then run segmentation, then create a mask from the segmentation. This flow is incredibly slow and wastes a lot of time.

Request

When I upload an image, I would like for the mask editing region (mark in red in the image above), to just have the uploaded image displayed with an empty mask. This way we can use the inpainting/cleaner features with our manually masked image without ever having to run the SAM model.

Alternatively, if there already exists a mask when a new image is uploaded, then it may be nice to keep the previous image's mask loaded. The mask would of course need to be scaled to the new image. You could also choose to just reset the mask if the image sizes are different. Which ever you'd prefer to implement.

Additional Notes

It looks like the input_image_upload event handler is expected to return the selection mask as its 2nd variable so I would expect it to just be a 2D numpy array of bools but it appears to be a uint8 array of the same shape as the input image (including depth channel). At some point, however, this return is being converted into a gradio Image but I can't seem to find where that is happening. Is there a possible edge case in here where the sam_image and sel_mask parameters to this handler are passed as NoneType causing the handler to return numpy arrays which would in turn throw an error later on?

Also it seems like the Clear button in the mask image editor doesn't work. I'm not sure if this just needs to be hooked up or if more needs to be done but it isn't currently clearing the mask.

[Uminosachi]

I have made a revision based on your suggestion. Now, when uploading an image, an empty mask will be displayed in the mask area at the bottom right, allowing you to edit the mask immediately. If a mask already exists and the size of the input image and mask match, the mask will be retained. If they do not match, the mask will be cleared.

Please update the repository sd-webui-inpaint-anything and try it out.

[Uminosachi]

Is there a possible edge case in here where the sam_image and sel_mask parameters to this handler are passed as NoneType causing the handler to return numpy arrays which would in turn throw an error later on?

If the input is of NoneType, there should be no problem with returning an np.ndarray. However, if the input is an instance such as dict type, the function will need to return the result using gr.update().

[Sj-Si]

Tested the changes and they appear to be working as expected. Thank you for the fix! :)

For future readers, these changes were made in the following commits:

• 4f0d8eba6ee46b9a2b79abeff7a2b124b024918d
• 067038bd18dd2b5ea93f55622670b5315051d422