andreszs
commented
1 month ago The controlnet models behaviour with SDXL is absoutely erratic. I only managed to avoid command line errors with these:
- openposeXL2-rank256
- thibaud_xl_openpose_256lora
As outlined in this reddit, controlnet in SDXL works more as guidelines that can be easily ignored depending on the prompt, regardless of their weight.
Anyway, this probably has nothing to do with Comfyui itself, meaning we are not the in the right place to discuss SDXL controlnet.
Hello,
SDXL and Pony are notorious for not being very good with controlnets, so I keep trying new ones.
I came accross this recently: https://civitai.com/models/136070/controlnetxl-cnxl and tried some of them.
I consistently get errors in the Apply ControlNet(Advanced) node with some authors, for instance, with Hetaneko's controlnet, I get the error
!!! Exception during processing!!! 'NoneType' object has no attribute 'copy' Traceback (most recent call last): File "C:\Users\maveyyl\AppData\Roaming\StabilityMatrix\Packages\ComfyUI\execution.py", line 151, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) File "C:\Users\maveyyl\AppData\Roaming\StabilityMatrix\Packages\ComfyUI\execution.py", line 81, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) File "C:\Users\maveyyl\AppData\Roaming\StabilityMatrix\Packages\ComfyUI\execution.py", line 74, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) File "C:\Users\maveyyl\AppData\Roaming\StabilityMatrix\Packages\ComfyUI\nodes.py", line 790, in apply_controlnet c_net = control_net.copy().set_cond_hint(control_hint, strength, (start_percent, end_percent)) AttributeError: 'NoneType' object has no attribute 'copy'
With SargeZT's controlnets, I get the error
!!! Exception during processing!!! Error(s) in loading state_dict for Adapter: Missing key(s) in state_dict: "body.6.block1.weight", "body.6.block1.bias", "body.6.block2.weight", "body.6.block2.bias", "body.7.block1.weight", "body.7.block1.bias", "body.7.block2.weight", "body.7.block2.bias". Traceback (most recent call last): File "C:\Users\maveyyl\AppData\Roaming\StabilityMatrix\Packages\ComfyUI\execution.py", line 151, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) File "C:\Users\maveyyl\AppData\Roaming\StabilityMatrix\Packages\ComfyUI\execution.py", line 81, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) File "C:\Users\maveyyl\AppData\Roaming\StabilityMatrix\Packages\ComfyUI\execution.py", line 74, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) File "C:\Users\maveyyl\AppData\Roaming\StabilityMatrix\Packages\ComfyUI\nodes.py", line 705, in load_controlnet controlnet = comfy.controlnet.load_controlnet(controlnet_path) File "C:\Users\maveyyl\AppData\Roaming\StabilityMatrix\Packages\ComfyUI\comfy\controlnet.py", line 388, in load_controlnet net = load_t2i_adapter(controlnet_data) File "C:\Users\maveyyl\AppData\Roaming\StabilityMatrix\Packages\ComfyUI\comfy\controlnet.py", line 547, in load_t2i_adapter missing, unexpected = model_ad.load_state_dict(t2i_data) File "C:\Users\maveyyl\AppData\Roaming\StabilityMatrix\Packages\ComfyUI\venv\lib\site-packages\torch\nn\modules\module.py", line 2041, in load_state_dict raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format( RuntimeError: Error(s) in loading state_dict for Adapter: Missing key(s) in state_dict: "body.6.block1.weight", "body.6.block1.bias", "body.6.block2.weight", "body.6.block2.bias", "body.7.block1.weight", "body.7.block1.bias", "body.7.block2.weight", "body.7.block2.bias".