IllusionDiffusion Colab - More options

kubinka0505 commented 1 year ago

Could the following options be implemented into the following sections?

Argument parser:

All settings in the Settings tab

This image can be interpreted as:

!python src/app.py --share --model "monster-labs/control_v1p_sd15_qrcode_monster" \
 --default-stable-diffusion-model "runwayml/stable-diffusion-v1-5" \
 --output-directory "/content/diffusionmagic/results" \
 --output-format png \
 --save-separated --enable-low-vram

!python src/app.py -s -m "monster-labs/control_v1p_sd15_qrcode_monster" \
 -dsdm "runwayml/stable-diffusion-v1-5" \
 -o "/content/diffusionmagic/results" \
 -f png \
 -ss -lvram

Gradio Web UI

Image height and width I have seen the lines responsible for it in the other files, but I don't see them implemented in the IllusionDiffusion notebook. https://github.com/rupeshs/diffusionmagic/blob/7b0571e0f6e5cbaeafdfc6e7fcc8066001ac559c/src/frontend/web/image_inpainting_ui.py#L43-L48
Importing options from the json files. Original SD webui notebook had this feature and in my opinion it would increase workflow. Even in Colab, user could change settings by #@params and writing a wrapper using the json module.

This image can be interpreted as:

{
    "prompt": {
        "positive": "",
        "negative": "low quality,bad, deformed, ugly, bad anatomy"
    },
    "quality": {
        "inference_steps": 20,
        "guidance_scale": 7.5,
        "sampler": "EulerAncestralDiscrete"
    },
    "seed": {
        "value": -1,
        "use_random": true
    },
    "controlnet": {
        "start": 0.0,
        "end": 1.0
    }
}

I would love to see these changes coming in - I could make a fork, edit and then do a pull request, but there are several files in with the similiar content listed in the src/frontend/web directory.

rupeshs commented 1 year ago

@kubinka0505

Argument parser

It can be extended to contain all main settings it was(model arg) mainly added to switch models in colab.

Gradio Web UI

Not sure about its performance with high resolutions.
What is the actual use of JSON settings, do you have an example or colab link? Also, low VRAM is not supported for illusion diffusion. Feel free to create PR, I'll check that.

kubinka0505 commented 1 year ago

@rupeshs

It can be extended to contain all main settings it was(model arg) mainly added to switch models in colab.

Yes, that's what I meant - the model could be changed using the dropdown menu:

DefaultModel = "stabilityai/stable-diffusion-xl-base-1.0" #@param ["stabilityai/stable-diffusion-2-1-base", "stabilityai/stable-diffusion-2-inpainting", "stabilityai/stable-diffusion-2-depth", "timbrooks/instruct-pix2pix", "runwayml/stable-diffusion-v1-5", "runwayml/stable-diffusion-inpainting", "CompVis/stable-diffusion-v1-4", "prompthero/openjourney-v2", "pcuenq/pokemon-lora", "sayakpaul/sd-model-finetuned-lora-t4", "prompthero/openjourney-v4", "lllyasviel/sd-controlnet-canny", "lllyasviel/sd-controlnet-mlsd", "lllyasviel/sd-controlnet-normal", "lllyasviel/sd-controlnet-hed", "lllyasviel/sd-controlnet-openpose", "lllyasviel/sd-controlnet-depth", "lllyasviel/sd-controlnet-scribble", "lllyasviel/sd-controlnet-seg", "stabilityai/stable-diffusion-xl-base-1.0", "warp-ai/wuerstchen"]"

Not sure about its performance with high resolutions.

It could enable panoramic or portrait modes of the pictures. width and height could be sliders with limited values, which the user will have to control responsibly.

What is the actual use of JSON settings, do you have an example or colab link?

SD WebUI loaded settings from the json files, such as prompt, negative_prompt etc. I forgot how it was loaded, but as an example I can say that the user wants to change the negative prompt to more extended, but not to be forced with default one on each launch.

In fact, it used styles.csv file, such as this one.

Feel free to create PR, I'll check that.

I cannot, as I do not know which file is directly responsible for the IllusionDiffusion. As I stated, many of them contain very similar contents, so I do not know which to edit.

A considerable idea would be to create a new branch dedicated to the IllusionDiffusion, but I guess it would cost time.

rupeshs / diffusionmagic