search

papuSpartan / stable-diffusion-webui-distributed

Chains stable-diffusion-webui instances together to facilitate faster image generation.
181 stars 14 forks source link

Compatibility with SD Forge (stable-diffusion-webui-forge) #25

Open gshawn3 opened 8 months ago

gshawn3 commented 8 months ago

First of all, thank you for this incredible extension. It seems very well designed and worked right away with very little configuration needed.

Assuming this is not already in the works, have you considered making the extension compatible with lllyasviel/stable-diffusion-webui-forge? That distribution is quite a bit faster than automatic1111, especially for lower RAM GPUs (50-75% speedup) which makes it a great platform to use on worker nodes.

As far as I can tell most everything is already working well, but there are some incompatibilities with ControlNet. One quick fix was to change the expected name from UiControlNetUnit to ControlNetUnit in distributed.py so that the ControlNet units are detected. After making that small change, a master server can now successfully send jobs that rely on ControlNet to workers running automatic1111. But there are still some issues sending those jobs to workers running on SD Forge, which I have not been able to figure out.

Anyway, I just wanted to see if this was on your radar. Thanks again for your hard work on this!

papuSpartan commented 8 months ago

Hey glad you're enjoying it so far and I appreciate you looking into making it better! I have thought about forge compatibility but haven't taken the time yet to make a forge install for actually doing this. I think I may try and do that this weekend. Currently I don't think sdwui has an api route for just getting the version (/config works for this but only on loopback?) so I doubt that forge does either. I could prob do something hacky to see if its forge but I may look into pushing something upstream for this.

If you notice anything else weird happening on forge definitely let me know.

gshawn3 commented 8 months ago

Adetailer is another popular extension that leverages ControlNet and recently added support for Forge.

It might be worth looking at the way that they are checking which version of ControlNet is available. The following two commits seem relevant: https://github.com/Bing-su/adetailer/pull/517/commits/d6f62d5e9d985193245fb307ffbcde0fa1829353 and https://github.com/Bing-su/adetailer/pull/517/commits/763278ca3f1ae8ac576eae9890f573409478ce1d.

Hope this helps!

papuSpartan commented 8 months ago

So far I have been able to reproduce the issue you've mentioned with not being able to direct forge instances using cni (controlnet integrated). Something seems to be going haywire with the cn payload im sending as what seems to be part of one of cni's caching mechanism is not being initialized? Here self.current_params[i] results in a KeyError (even with only 1 cn unit passed) as self.current_params is empty at the time of the call. So far I'm unsure as to why this is happening as I'm unfamiliar with cn's inner-workings. I did find in lllyasviel/stable-diffusion-webui-forge#259 that some have experienced this same issue before but only when forgetting to include a model name.

Still looking into it.