Feedback for “Models”

[DieserBobby]

Hello, I had bought the installer for IOPaint because I am not into this pip-installing-method. Everything went fine so far, but I do not fully understand the way models are implemented.

Specially how to download and install erase models... I do not understand how to install ZITS for example.

I have successfully created a custom model folder, and in this folder I created 2 folders "stable_diffusion" and "stable_diffusion_xl". I noticed, that putting an XL-model into the "stable_diffusion_xl" will make it appear in the menu in IOPaint - so far so good.

But this method does not work for Erase-Models, because 1 : I would not know where exactly to put them and 2: There is not the typical download button on Huggingface to download the .safetensors-file.

I have seen that there are 3 drop-downs about models in the config-UI... I think I did not yet fully understand how these work together. Recommended Models / Downloaded Models / Current Model. I have difficulties in understanding the system how it works.

Could you help please? maybe some more detailed step by step documentation for non-coders? I came to IOPaint through NerdyRodents YT-Video: IOPaint (Lama-Cleaner): A FREE & Simple Inpainting / Outpainting App! But I couldn`t understand installation, so I bought the installer. best regards and thanks for the cool application... Bobby

[Sanster]

Thank you for supporting my open-source project. For all Erase-Models, there is no need for manual downloading. Once the model is configured, it will automatically download upon service startup.

custom model folder

I have successfully created a custom model folder, and in this folder I created 2 folders "stable_diffusion" and "stable_diffusion_xl".

The custom model folder is for use with the diffusion model files in single file format, such as the sd-v1-5-inpainting.ckpt file shown below, or the single file models downloaded from civitai. For model files in the diffusers format, there is no need for manual downloading. As shown in the figure below, as long as the model is configured as runwayml/stable-diffusion-inpainting, IOPaint will automatically download all the files upon startup.

Recommended Models / Downloaded Models / Current Model

When saving the configuration file, the model name in the Current Model will be saved in the configuration file, and this model will serve as the model upon service startup. "Recommended Models" or "Downloaded Models" dropdown menus are just for the convenience of switching the configuration of the Current Model.

• Recommended Models displays some commonly used models, When users haven't downloaded any models yet, they can select models from here.
• Downloaded Models will show all the Erase-Models, single-file diffusion models, and diffusers format diffusion models that have been downloaded.

[DieserBobby]

Hi,

thanks so much for your detailed answer. I understand a lot more now, also your screenshot was very helpful to progress in understanding.

Some problem still remains though... I hope you can give me some direction to solve...

I wanted to choose the ZITS_inpainting for example. But I don't know how.

My settings look like this:

my config looks like this:

To integrate models I have to manually set them in the config and then restart?

And the tab "other Diffusion"?

I cannot see the ZITS... is this included somewhere else? I suppose, that as soon I understand how to integrate ZITS I will understand the process, or maybe something went wrong installing the package?

I can see that creating the software has been a lot of work, really impressive and flexible with so many models. I am looking forward using it.

I am using NVIDIA GeForce RTX 3060 Do you recommend changing some of the default settings? (I have seen the word cuda in the config )

thanks for your support Bobby

[DieserBobby]

Hi, I am still stuck with the same situation (described above) I do not know how to choose ZITS for inpainting... it looks like I dont have it. Hope to receive some additional information.

[Sanster]

Apologies for the late reply, if you wish to use the ZITS model, you can manually input zits in the "Current Model".

Other Erase Models are the same, for example https://www.iopaint.com/models/erase/fcf, you can manually input fcf in the Current Model.

The reason why the ZITS model does not appear in the Recommended Models is because I found that it seems to have little advantage compared to lama (lama is more versatile). Feel free to test it in your scenario.