[Reference Control Type] "Allow preview" and "Preview as input" checkbox should not appear

[Talae06]

Is there an existing issue for this?

• [X] I have searched the existing issues and checked the recent builds/commits of both this extension and the webui

What happened?

When using one of the "reference" control type (whichever preprocessor you then choose after), the checkboxes "Allow peview" and, more importantly, "Preview as input" stay present, thus making the user think that it's ok to have them ticked.

That misconception is reinforced by the fact that the icon "Run preprocessor" also stays present, and acts like usual. If used, it automatically ticks the "Allow preview" checkbox and indeed runs the preprocessor and make the preview appear.

Considering all this, it seems only natural, for a user who would have proceeded this way, to also tick the "Preview as input" checkbox, maybe thinking that, as for other control types, ticking it might save a few seconds when launching the picture genration, since the preprocessor has already been run.

The problem is, that if "Use preview as input" is indeed ticked, the reference controlnet is totally and completely ignored.

An error message does appear in the console when launching the picture generation, but is not really explicit, since it concludes with "You have not selected any ControlNet model" (see below for full transcript).

I'm aware that the documentation states "To use, just select reference-only as preprocessor and put an image" and thus implicitly tells not to use these two checkboxes. But I still think that having them stay present when the control type is set to Reference, induces problematic confusion, whereas they, in that situation and as far as I can tell, don't serve any purpose.

Steps to reproduce the problem

1. Enable ControlNet
2. Select "Reference" in "Control Type"
3. Drop a reference image
4. Click the "Run preprocessor" icon, thus automatically ticking "Allow Preview" (or tick it manually and then click the icon)
5. Manually tick "Preview as input"
6. Generate the picture

What should have happened?

Since, unless I'm mistaken, the checkboxes "Allow preview" and more importantly "Preview as input", as well as the "Run preprocessor" icon, serve no purpose to the user if the control type is set to "Reference", and can actually be problematic, they should disappear when the control type is set to "Reference".

Commit where the problem happens

webui: 1.5.1 controlnet: 1.1.238 and earlier.

What browsers do you use to access the UI ?

Mozilla Firefox

Command Line Arguments

I tried with only "--autolaunch" as well as with my usual "--skip-version-check --opt-sdp-no-mem-attention --api --theme light --autolaunch"

List of enabled extensions

I disabled all extensions for these tests.

Console logs

*** Error running process: D:\Stable Diffusion\Automatic1111\extensions\03 ControlNet\scripts\controlnet.py
    Traceback (most recent call last):
      File "D:\Stable Diffusion\Automatic1111\modules\scripts.py", line 519, in process
        script.process(p, *script_args)
      File "D:\Stable Diffusion\Automatic1111\extensions\03 ControlNet\scripts\controlnet.py", line 753, in process
        model_net = Script.load_control_model(p, unet, unit.model, unit.low_vram)
      File "D:\Stable Diffusion\Automatic1111\extensions\03 ControlNet\scripts\controlnet.py", line 306, in load_control_model
        model_net = Script.build_control_model(p, unet, model, lowvram)
      File "D:\Stable Diffusion\Automatic1111\extensions\03 ControlNet\scripts\controlnet.py", line 316, in build_control_model
        raise RuntimeError(f"You have not selected any ControlNet Model.")
    RuntimeError: You have not selected any ControlNet Model.

Additional information

No response

[lllyasviel]

I reworked the terrible current logics about preview in 1.1.305. See if that is much better now

[Talae06]

It does seem much better indeed ! Thanks a lot :)

[SAC020]

Hi! "preview as input" is no longer available / is missing in openpose, and it is needed there because there is an preview editor available (I, personally, use it a lot)

Controlnet version 9af9d6b9

[Talae06]

@SAC020 From what I gather, the new way it works automatically uses the preview for input if a preview has indeed been generated. And the "Edit" button is still there for OpenPose :

And as far I can tell, the modifications you make are indeed taken into account :

[SAC020]

Thank you, but, then, how would I know for which controlnets does it work automatically, and for which doesn't / ignores the preview? Is it only for openpose?

Using previews also had a (slight) speed benefit for multiple repeated generations.

[Talae06]

I'm guessing it now works this way for all control types, I don't see why it would vary, but only @lllyasviel can confirm it I guess.

As for the second point, again, my understanding is that it automatically uses the preview for input if a preview has been generated, so the time benefit should be the same as before. To try and confirm it, I just ran a quick test using one of the more time-consuming control type/preprocessor : Seg with seg_ufade20k. I generated the preview first, and then launch the picture generation. The console messages do seem to confirm it.

Having the preview generated first :

Without having the preview generated :

[huchenlei]

Now the option is reserved only for openpose editor.

For other preprocessors, the repeated generation will go through the preprocessor cache so there should not be overhead regenerating the preprocessor map.

You can check whether the generation is going through cache by setting log level to DEBUG.

Note: Going through cache is not the same as use preview. For preprocessors other than openpose, if you modify the generated image, the effect will not be reflected in generation.

[Talae06]

That's good to know, thanks !

[SAC020]

Thank you

[Prince-Mars]

For other preprocessors, the repeated generation will go through the preprocessor cache so there should not be overhead regenerating the preprocessor map.

Please, return Preview as input! I'd like to be able to use "Preview as input" without the original image, which I've noticed has an effect.