Replacer is an extension for AUTOMATIC1111/stable-diffusion-webui. The goal of this extension is to automate objects masking by detection prompt, using sd-webui-segment-anything, and img2img inpainting in one easy to use tab. It also useful for batch inpaint, and inpaint in video with AnimateDiff
You also can draw your mask instead of or in addition to detection, and take advantage of convenient HiresFix option, and ControlNet inpainting with preserving original image resolution and aspect ratio
If you find this project useful, please star it on GitHub!
Extension
-> Available
-> click Load from
and search "sd-webui-segment-anything"extensions/sd-webui-segment-anything/models/sam
lcm-lora-sdv1-5.safetensors
, put into models/Lora
. Or if you have already lcm lora, then change hires suffix in the extension optionsExtension
-> Available
-> click Load from
and search "Replacer". For AMD and Intel GPUs, and maybe something else, you need to enable CPU for detection in Replacer's settings. Go to Settings
-> Replacer
and enable itIf you don't want to use Video feature, that's all for you. Further steps are for Video:
Extension
-> Available
tab and find them thereextensions/sd-webui-animatediff/model
directorymodels/ControlNet
directorymodels/ControlNet
directory. Then you can select SparseCtrl module in ControlNet extension. The rgb one requires "none" preprocessorSAM-HQ are the best for me. Choose it depending on your vram. Sum this model size with dino model size (694MB-938MB)
1. [SAM](https://github.com/facebookresearch/segment-anything) from Meta AI. - [2.56GB sam_vit_h](https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth) - [1.25GB sam_vit_l](https://dl.fbaipublicfiles.com/segment_anything/sam_vit_l_0b3195.pth) - [375MB sam_vit_b](https://dl.fbaipublicfiles.com/segment_anything/sam_vit_b_01ec64.pth) 2. [SAM-HQ](https://github.com/SysCV/sam-hq) from SysCV. - [2.57GB sam_hq_vit_h](https://huggingface.co/lkeab/hq-sam/resolve/main/sam_hq_vit_h.pth) - [1.25GB sam_hq_vit_l](https://huggingface.co/lkeab/hq-sam/resolve/main/sam_hq_vit_l.pth) - [379MB sam_hq_vit_b](https://huggingface.co/lkeab/hq-sam/resolve/main/sam_hq_vit_b.pth) 3. [MobileSAM](https://github.com/ChaoningZhang/MobileSAM) from Kyung Hee University. - [39MB mobile_sam](https://github.com/ChaoningZhang/MobileSAM/blob/master/weights/mobile_sam.pt)
FastSAM and Matting-Anything aren't currently supported
First, grounding dino models detect objects you provided in the detection prompt. Then segment anything model generates contours of them. And then extension chooses randomly 1 of 3 generated masks, and inpaints it with regular inpainting method in a1111 webui
When you press the "Apply hires fix" button, the extension regenerates the image with exactly the same settings, excluding upscaler_for_img2img. Then it applies inpainting with "Original" masked content mode and lower denoising but higher resolution.
1. [Lama cleaner as masked content](https://github.com/light-and-ray/sd-webui-lama-cleaner-masked-content) 1. [Image Comparison](https://github.com/Haoming02/sd-webui-image-comparison) 1. [ControlNet](https://github.com/Mikubill/sd-webui-controlnet) 1. [AnimateDiff](https://github.com/continue-revolution/sd-webui-animatediff) 1. [ar-plusplus](https://github.com/altoiddealer/--sd-webui-ar-plusplus) (maybe works with "Aspect Ratio and Resolution Buttons" forks) 1. Other extension scripts which doesn't have control arguments, e.g. [Hardware Info in metadata](https://github.com/light-and-ray/sd-webui-hardware-info-in-metadata), [NudeNet NSFW Censor](https://github.com/w-e-w/sd-webui-nudenet-nsfw-censor), built-in **Hypertile**