How to use this in Automatic1111 Stable Diffusion Web UI (for SD 1.5, XL etc)

[SMUsamaShah]

How to use this in Automatic1111 Stable Diffusion Web UI? Can you add steps to readme if its as simple as changing a few lines?

I want to use it with SD 1.5 mainly because it seems to give SD 1.5 a new life (to gen high res images)

[Luke2642]

You can't do the one line thing in Auto1111.

HiDiffusion is built on the diffusers framework. The "one line" thing only works on UIs that use the diffusers framework.

A quick google didn't turn up many options, just https://github.com/invoke-ai/InvokeAI which is still under active development, and an old one, https://github.com/abhishekkrthakur/diffuzers

There's a couple of possible places it could go in invokeAI:

InvokeAI\invokeai\backend\model_manager\load\model_loaders\stable_diffusion.py InvokeAI\invokeai\backend\model_manager\load\model_loaders\generic_diffusers.py

But diffuzers looks super simple:

diffuzers\diffuzers\text2img.py

Best of luck!

[CCpt5]

ComfyUI developer Kijai added a simple SD1.5 wrapper implementation of it in his ELLA node: https://github.com/kijai/ComfyUI-ELLA-wrapper. See commit from April 24: https://github.com/kijai/ComfyUI-ELLA-wrapper/commits/main/

However, I've had (and confirmed w/ him that he has had) a reported issue w/ generating non-square resolutions. The fix pushed yesterday which closed the thread here on it did not resolve it unfortunately: (Closed case here):https://github.com/megvii-research/HiDiffusion/issues/6

Also per that link the SD.Next developer Vlad is working on incorporating it on his (formerly fork) of A1111. I think it's only in his dev branch for now but that might be the UI to have full implementation first.

Even if the non-square problem is worked out,, Kijai's comfy wrapper wouldn't support SDXL since ELLA doesn't. Aparently there are a lot of things to consider when making a node for this specifically. I hope the developers here can make nodes/extensions happen!!

[ShenZhang-Shin]

Is the non-square resolution problem still happens? Man, let me know what model and resolution you are using.

[CCpt5]

Is the non-square resolution problem still happens? Man, let me know what model and resolution you are using.

I ran the error I kept getting last night after trying the latest commit you pushed through Claude Opus. This was their reply on it:

A lot of us are very excited about your work here, so I hope we can get it running in Comfy and/or A111 (automatic).

[ShenZhang-Shin]

you run hidiffusion on ComfyUI？ I see the problem is the size mismatch between hidden_states and res_hidden_states. But I still don’t know the image resolution. If I know it, I can reproduce the problem on my device and solve it easily.

[CCpt5]

you run hidiffusion on ComfyUI？ I see the problem is the size mismatch between hidden_states and res_hidden_states. But I still don’t know the image resolution. If I know it, I can reproduce the problem on my device and solve it easily.

Developer Kijai added a very limited wrapper implementation of it in a node he created for ELLA. His wrapper only supports the SD1.5 but implemented here: https://github.com/kijai/ComfyUI-ELLA-wrapper/commit/457c9b8a1ea5d98ba9c840481a7bf5552fdc7f20

Can find a bunch of the comfy developers in the Banodoco server on Discord (AnimateDiff etc): https://discord.gg/r22ZNk3J

I had issues w/ 1080x720 for example. Don't know exactly what I had been trying when I got that error above. I asked the developer of the node (Kijai) and he replied that he had issues even after the latest commit (fix) also. Here are the relevant convos:

[florestefano1975]

I created this nodes to start testing this technology on ComfyUI: https://github.com/florestefano1975/ComfyUI-HiDiffusion

[CCpt5]

I created this nodes to start testing this technology on ComfyUI: https://github.com/florestefano1975/ComfyUI-HiDiffusion

Thanks for a way to trial. I assume you can only use the base model it pulls w/ your wrapper here?

[Remember2015]

I created this nodes to start testing this technology on ComfyUI: https://github.com/florestefano1975/ComfyUI-HiDiffusion

So, does this plugin work properly now?

[xiao2mo]

you run hidiffusion on ComfyUI？ I see the problem is the size mismatch between hidden_states and res_hidden_states. But I still don’t know the image resolution. If I know it, I can reproduce the problem on my device and solve it easily.

Developer Kijai added a very limited wrapper implementation of it in a node he created for ELLA. His wrapper only supports the SD1.5 but implemented here: kijai/ComfyUI-ELLA-wrapper@457c9b8

Can find a bunch of the comfy developers in the Banodoco server on Discord (AnimateDiff etc): https://discord.gg/r22ZNk3J

I had issues w/ 1080x720 for example. Don't know exactly what I had been trying when I got that error above. I asked the developer of the node (Kijai) and he replied that he had issues even after the latest commit (fix) also. Here are the relevant convos:

Hi，Exactly the same question here. I just use the code for SDXL inference not the ComfyUI. Have u solved the problem yet?

[xiao2mo]

-> 1710 hidden_states = torch.cat([hidden_states, res_hidden_states], dim=1) 1712 if self.training and self.gradient_checkpointing: 1714 def create_custom_forward(module, return_dict=None):

RuntimeError: Sizes of tensors must match except in dimension 1. Expected size 90 but got size 45 for tensor number 1 in the list.

I just don't know why the feature map is half

[greasebig]

is there any originally implemented extension that can load A1111 webui model?

[ShenZhang-Shin]

-> 1710 hidden_states = torch.cat([hidden_states, res_hidden_states], dim=1) 1712 if self.training and self.gradient_checkpointing: 1714 def create_custom_forward(module, return_dict=None):

RuntimeError: Sizes of tensors must match except in dimension 1. Expected size 90 but got size 45 for tensor number 1 in the list.

I just don't know why the feature map is half

You can try the latest hidiffusion version and the problem would be fixed

pip3 install hidiffusion -U