Connected to auto1111 server, but control net models not showing up

[filliptm]

im connected to the backend of auto1111 and i have control net and the models installed, and they are not popping up in the aitools UI

[SethRobinson]

Hmm, happy to fix it if I can. Couple questions:

1. If you use the auto1111 webUI interface directly (without my app), do the models show up, are selectable, and work? (I assume they exist on the server part in extensions/sd-webui-controlnet/models

2. Are you running both on Windows on the same computer? The only setup I've tested with so far is client on windows, server on linux on a different computer so far, so if your setup is different I'll try it myself that way to test. Wouldn't be the first time / vs \ in file names screwed me up.

3. Are you using the latest sd-webui-controlnet version (you can update it with "git pull" in its extension directory, or use the webUI interface to update)? I was using the newest as of yesterday for my tests, old versions wouldn't work due to API differences.

[filliptm]

thanks for the response! im in love with your app.

The control net models do show up in my auto1111. My custom models show up in your app, but the control net ones do not.

im running windows 11 both the backend and your app, same computer

yes im using the latest update of control net!

again thanks for your work =)

[SethRobinson]

Thanks!

Ok, I did some tests using your setup with a clean install. It did work for me - although I did have some problems at first:

• I couldn't get it to work at all until I edited webui-user.bat (the file I double click to run the server locally) and added --api --xformers

I don't know why I need --xformers, but without that, the server itself crashes when http://localhost:7861/docs#/default/get_config_sdapi_v1_options_get is tested - (500: Internal error) this causes AI Tools to not connect at all.

• When I clicked the checkbox to enable ControlNet in AI Tools, it gave an incorrect warning, that's my bug, it shouldn't warn there, you can ignore it

• For me, it did successfully list all the ControlNet models, I can't recreate your error

• in the server dos window, I noticed every time I use ControlNet I did get this error though:

Loaded state_dict from [D:\pro\stable-diffusion-webui\extensions\sd-webui-controlnet\models\control_sd15_depth.pth]
Error running process: D:\pro\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py
Traceback (most recent call last):
  File "D:\pro\stable-diffusion-webui\modules\scripts.py", line 386, in process
    script.process(p, *script_args)
  File "D:\pro\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 612, in process
    else self.build_control_model(p, unet, model, lowvram)
  File "D:\pro\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 471, in build_control_model
    network = network_module(
  File "D:\pro\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\cldm.py", line 63, in __init__
    config = OmegaConf.load(config_path)
  File "D:\pro\stable-diffusion-webui\venv\lib\site-packages\omegaconf\omegaconf.py", line 187, in load
    with io.open(os.path.abspath(file_), "r", encoding="utf-8") as f:
FileNotFoundError: [Errno 2] No such file or directory: 'D:\\pro\\stable-diffusion-webui\\models\\cldm_v15.yaml'

It seems to be looking for cldm_v15.yaml in the wrong directory. In settings, I changed it to the full path: ( D:\pro\stable-diffusion-webui\extensions\sd-webui-controlnet\models\cldm_v15.yaml ) and the error was gone and it was able to properly run (I noticed it downloaded something from huggingface) This is probably a bug in the extension itself.

• Strangely, when DDIM is selected as the sampler, it uses the incorrect steps. If set to 50, it might do 49 or 40, weird. Probably a bug in the extension.

• There is a ControlNet option in Settings called "[ ] Allow other script to control this extension", you might try turning that on, but it works for me with it off so probably doesn't matter.

So to recap: It does work for me, but there are definitely some issues, some which may be caused by my python/pip windows setup, or problems with the extension itself, which might be fixed in time. I would try running with --api --xformers if you haven't, hopefully you can get it running soon.

[SethRobinson]

Oh, one more thing I just thought of, do you have any spaces or weird symbols in the directory/install path or any model filenames? That might break things.

[filliptm]

Hmm thats interesting. I have --api and --xformers on of course

i'll record a video of the probem and the log and see if anything sticks out to you

[filliptm]

my models have an "@" symbol in the name, but not in the control net models, which is where the problem is happening

[SethRobinson]

Oh, and inside of AI Tools.exe, click "Show/hide log" and check for anything that looks weird there when connecting.

[SethRobinson]

Any chance you are you running from a network drive, like //drivename instead of c:\ Sorry for the million messages. :)

[filliptm]

So maybe my version isn't an ai tools variant?

[SethRobinson]

That's true, but it's ok, the ControlNet stuff should work fine anyway. The only thing missing will be the "Remove background" button won't work. The paintball game test and webcam test won't work right because of that, but everything else should be ok.

[filliptm]

damn, yeah im not sure. I even toggled on the setting to let other scripts use the control net

[SethRobinson]

Hmm. It seems like the server is returning zero models for some reason. It should look like this:

Oh, we can test the API directly on the server. Visit this link on your server: http://localhost:7860/docs#/default/model_list_controlnet_model_list_get

Then click "Try it out" and then "Execute".

You should see a list of models in this format:

I'm guessing the list is blank or something?

[filliptm]

Correct its blank

[SethRobinson]

Ahh ok. Hmm, ok, this means it has to be a problem with your install, maybe a bug in the extension itself, something to do with your setup that doesn't affect mine. Weird that would be blank but the models do show up for you elsewhere in the auto1111 webui, possibly it's scanning the wrong directory or something. Probably won't work, but one thing to try is running it directly from the command line (cd into the correct dir) with "call webui-user.bat" instead of double clicking it with fileexplorer, or vice-versa.

I'm going to release a V0.72 version soonish (today maybe), I'll throw in a thing that will fake-populate the list if it detects this problem, it might allow you to work around the problem.

[filliptm]

Alright I can give that a try

What directory is it looking for files? is it possible to tell your app to look in a different directory for models? Everything is set pretty much default and i only have 2 models in there

[SethRobinson]

That's exactly the same setup as mine:

I removed all but the two models you have to make it a better test, and that still worked properly for me. (those two models did show up ok)

This error is happening before my app is loaded, but my app can fake like it found them, but it will depend on if the server can find them or not when it processes the image, so my 0.72 work-around might not work.

Might want to open an issue here. Stuff is really in flux with the extension right now, especially with its API. :)

PS: My app doesn't check directories at all for what server models exist, it just talks to the server via the API (which will give the list.. uh.. if it worked), this makes it possible to all work fine even if the server is remote.

[SethRobinson]

Note: I released a new version, I did add the workaround hack for "server is giving a list of nothing for the ControlNet models" issue, hopefully it works for ya. Also, try removing the dreambooth extension, it's been reported to screw with server API stuff somehow, it's the reason for the "Protocol Error" messages I think.