search

TheCodeSlinger / FacePop

A stable-diffusion-webui extension for enhancing faces in img2img generation.
3 stars 0 forks source link

readme

FacePop

(Version 1.2.2)

FacePop Logo FacePop Logo

Cropping Upscaling and BG Removal

Face Crop + Background Removal

An example of the low resolution image cropped, background removed, angle set for best generative img2img processing and first processing pass.

Face Alignment for Processing

Landmark Detection and Facial Upright Alignment

The red dots indicate the original position of the landmark detection. The image behind it is the face tilted to the most ideal upright angle for generative img2img processing.

Final Process Masking

Final pass masking using Inpainting mask, combines with Inpainting

For final image composite a robust mask inpainting mask is applied so it doesn't receive a second generative process, but the rest of the image does. This allows for very good results of generative blending with the rest of the image.

Introduction

FacePop is a robust extension for AUTOMATIC1111's Stable Diffusion Web UI that enhances image processing by detecting, enhancing, and managing faces within images. Leveraging advanced technologies like Mediapipe, MODNet, and ControlNet, FacePop streamlines tasks such as face detection, background removal, and image enhancement directly within your Stable Diffusion workflow.

Motivation

FacePop was developed as a solution to the limitations encountered with existing tools like Zoom Enhancer, eliminating the need for additional extensions such as Unprompted. While Zoom Enhancer provides basic facial zoom capabilities, FacePop offers more control and flexibility over facial detection and processing features. This extension not only enhances the zoom functionality but also integrates seamlessly with other popular plugins like ControlNet, ReActor, and After Detailer.

By leveraging these integrations, FacePop allows users to customize their image processing workflows extensively, ensuring that each facial enhancement task can be fine-tuned to meet specific requirements. Whether it's for facial feature refinement, background removal, or advanced image manipulation techniques, FacePop provides the necessary tools to achieve better results. This approach ensures that users have greater control over their image enhancements, making FacePop a valuable tool for anyone looking to improve their Stable Diffusion experience.

What it Does

FacePop simplifies facial image processing within the Stable Diffusion workflow by offering a straightforward and customizable solution. Here's how FacePop works:

  1. Face Detection:

    • Spotting Faces: Uses Mediapipe to accurately find and locate all the faces in your image.
    • Finding Landmarks: Identifies key points on each face to ensure they're properly aligned for the next steps.

  2. Face Cropping and Upscaling:

    • Cutting Out Faces: Each detected face is neatly cropped from the main image based on your settings for width, height, and padding.
    • Boosting Resolution: The cropped faces are then upscaled to your chosen dimensions, making them clearer and more detailed.

  3. Rotating Faces Upright:

    • Aligning Faces: Faces that are at odd angles or even upside down are rotated to an upright position. This ensures consistency and better quality during processing.
    • Consistent Processing: By standardizing the orientation, FacePop ensures that all faces, regardless of their original position, are enhanced uniformly.

  4. Separate Processing of Faces:

    • Focused Enhancements: Each upright and upscaled face is processed individually. This allows for targeted improvements like color correction, sharpening, and background removal using MODNet.
    • Independent Settings: You can adjust sampling steps, CFG scaling, and denoising levels specifically for face processing, giving you more control over the final look.

  5. Mask Creation and Integration:

    • Making Masks: After processing, FacePop creates masks around each enhanced face. This ensures that only the faces are affected during reintegration.
    • Seamless Blending: The masks help blend the enhanced faces back into the original image smoothly, avoiding any visible edges or mismatches.

  6. Final Image Composite and Processing:

    • Putting Faces Back: The enhanced and properly oriented faces are placed back into their original spots in the main image using the masks.
    • Final Touches: The entire image is then processed again, which might include additional enhancements or integrations with other plugins like ControlNet, ReActor, and After Detailer. This step ensures everything looks polished and cohesive.

Why FacePop?

With FacePop, you get:

By handling faces in a structured and efficient way, FacePop helps you achieve high-quality images with minimal effort. Whether you're tweaking a single portrait or handling multiple faces in a group shot, FacePop streamlines the process, making your image enhancements faster and more reliable.

Table of Contents

Features

Installation

Prerequisites

Before installing FacePop, ensure you have the following:

Install via Extensions Manager

  1. Open Stable Diffusion Web UI: Launch your Stable Diffusion Web UI as you normally would.

  2. Navigate to Extensions: Click on the Extensions tab in the sidebar.

  3. Install from URL:

    • Click on the Install from URL button.
    • Enter the repository URL: https://github.com/TheCodeSlinger/FacePop.git
    • Click Install.

    Install from URL

  4. Restart Web UI: After installation, you may be prompted to restart the Web UI. Follow the prompt to apply changes.

  5. Verify Installation:

    • Once restarted, navigate to the Extensions tab to ensure FacePop is listed.
    • Alternatively, check within the Img2Img interface for the FacePop panel.

Usage

  1. Open Img2Img Interface: Navigate to the Img2Img tab within the Stable Diffusion Web UI.

  2. Locate FacePop Panel: Scroll down to find the FacePop accordion or panel.

  3. Configure Settings:

    • Enable FacePop: Toggle the extension on.
    • Use MODNet: Enable background removal.
    • Enable Aggressive Face Detection: Enhance face detection capabilities.
    • Adjust Parameters: Set face width, height, padding, detection confidence, maximum faces, and more as per your requirements.

    FacePop Settings

  4. Process Image:

    • Upload or generate an image with faces.
    • Configure the desired settings in the FacePop panel.
    • Click Generate to process the image with FacePop enhancements.

  5. Review Results:

    • Processed images will display enhanced faces.
    • Check the output directory for individual processed face images and the final composite image.

Deny Scripts List

deny_scripts_list.txt is a crucial configuration file in the FacePop extension that allows users to specify which scripts should be ignored during specific processing stages. This ensures seamless integration and prevents potential conflicts between FacePop and other extensions or scripts within the Stable Diffusion Web UI.

Purpose

The primary purpose of the deny_scripts_list.txt file is to control the activation of certain scripts during different phases of the image processing workflow. By specifying scripts to be ignored, FacePop can operate without interference, ensuring optimal performance and stability.

File Structure

The deny_scripts_list.txt file is organized into sections, each corresponding to a different processing stage. Within each section, you can list the names of scripts that should be disabled during that particular stage.

; ignore any of these scripts when faces are being processed
[faces]
ADetailer

; ignore any of these scripts during final image composite
[final]
ReActor

NOTE: The main reason ReActor is excluded from final process is it seems to ingore inpaint masks, which the final composite image requires to prevent double processing of the face.

NOTE: ADetailer, we want to work but only on the final compsite image, not the upscaled cropped face. Some features in Adetailer work on other things like fingers and full body that may not be visible in just the face upscale. It hasn't been fully tested yet, and the final mask may prevent it from working as intended and I may need to come back to a solution for that if issues are submitted requesting this.

Screenshots

FacePop UI Panel Figure 1: FacePop panel within the Img2Img interface.

Before and After Figure 2: Comparison of an image before and after FacePop processing.

Dependencies

FacePop relies on several third-party libraries and tools to function effectively:

These dependencies are automatically handled during installation via the Extensions Manager. If you encounter issues, you can manually install them using pip:

pip install mediapipe modnet torch torchvision opencv-python gradio

Additional Resources

Model Files

FacePop utilizes pre-trained models for efficient and accurate face detection. Below are the details of the essential model files used, including their descriptions and licensing information.

1. deploy.prototxt

2. res10_300x300_ssd_iter_140000.caffemodel

3. MODNet

4. How to Obtain and Integrate the Model Files

All these are already included, you don't need to go donwload them unless you want to get them directly from the source.

  1. Download the Files:

  2. Place the Files in \stable-diffusion-webui\extensions\FacePop\scripts\:

License

MIT License

Copyright (c) 2024 TheCodeSlinger

Permission is hereby granted, free of charge, to any person obtaining a copy
...

Updates

[10-07-2024] Bug Fixes, Retooled lots of stuff, added Restore Eyes (version 1.2.0)

[09-24-2024] Fixed some Inpainting issues (version 1.1.2)

[09-23-2024] Inlcuded mediapipe in installer.py (version 1.1.1)

[09-23-2024] Bug Fixes and Feature integrations (version 1.1.0)

[09-19-2024] Original release (version 1.0.0)