kijai / ComfyUI-ELLA-wrapper

Simple wrapper to try out ELLA in ComfyUI using diffusers
Apache License 2.0
109 stars 7 forks source link

SDXL support? #15

Open MNeMoNiCuZ opened 6 months ago

MNeMoNiCuZ commented 6 months ago

Is such a thing possible for now?

I get this error in the ELLA Model Loader when I try to use an SDXL model:

Error occurred when executing ella_model_loader:

Trying to set a tensor of shape torch.Size([640, 640]) in "weight" (which has shape torch.Size([640, 640, 1, 1])), this look incorrect.

File "C:\AI\ComfyUI\execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\AI\ComfyUI\execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\AI\ComfyUI\execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\AI\ComfyUI\custom_nodes\ComfyUI-ELLA-wrapper\nodes.py", line 170, in loadmodel
set_module_tensor_to_device(unet, key, device=device, dtype=dtype, value=converted_unet[key])
File "C:\AI\ComfyUI\venv\Lib\site-packages\accelerate\utils\modeling.py", line 310, in set_module_tensor_to_device
raise ValueError(
kijai commented 6 months ago

Nope, they didn't release SDXL version of ELLA and have said they do not plan to.

MNeMoNiCuZ commented 6 months ago

Got it! Thanks for sharing. That's a shame :) But hey, this is cool as well! We can always switch models and use of a base for refinement :)

Thanks for the work, it worked flawlessly out of the box. I'm quite impressed by the results of their model, the prompt comprehension is very impressive!