ahrm / UnstableFusion

A Stable Diffusion desktop frontend with inpainting, img2img and more!
GNU General Public License v3.0
1.26k stars 86 forks source link

UnstableFusion

A Stable Diffusion desktop frontend with inpainting, img2img and more!

https://user-images.githubusercontent.com/6392321/191858568-0550f52d-e89c-4b37-aa07-23df605b4807.mp4

How to run locally?

  1. Install the dependencies (for example using pip). The dependencies include :
    • PyQt5, numpy, pytorch, Pillow, opencv-python, requests, flask, diffusers, transformers, protobuf, qasync, httpx

Package versions given in "requirements.txt" may, depending on environment, need to be supplemented by:

Note that if you want to run StableDiffusion on Windows locally, use requirements-localgpu-win64.txt

pip install -r requirements-localgpu-win64.txt

Note: On linux, if you encounter Could not load the Qt platform plugin "xcb" error, run this:

pip uninstall opencv-python     (solve a xcb compatibility issue)
pip install opencv-python-headless     (solve a xcb compatibility issue)
  1. Create a huggingface account and an access token, if you haven't done so already. Request access to the StableDiffusion model at CompVis/stable-diffusion-v1-4.

  2. Clone this repository and run python unstablefusion.py

How to run using colab servers?

  1. Install the dependencies (see the previous section)

  2. Open this notebook and run it (you need to enter your huggingface token when asked).

  3. When you run the last cell, you will be given a url like this:

colab_output - Copy copy this URL.

  1. Run python unstablefusion.py

  2. In the runtime section, select server and enter the address you copied in the server field. Like this:

using_server - Copy

How to use?

How to use advanced inpainting?

Admittedly, the UI for advanced inpainting is a little unintuitive. Here is how to works:

  1. You clear the part of the image that you want to inpaint (just like normal inpainting)
  2. Select the target box (again, like normal inpainting), but instead of clicking on the inpaint button, click on Save Mask button. From now on, the current mask and current selected box will be used for inpainting no matter how you change the box/image (until you press Forget Mask button)
  3. This means that you are free to edit the initial image as you please using other operations. For example, you can autofill the masked area using Autofill button, or manually paint the target area or paste any image from scratchpad. Since this initial image will be used to initialize the masked part, it will heavily affect the final result. Therefore by controlling this initial image, you can modify the final result to your will.