search

AUTOMATIC1111 / stable-diffusion-webui

Stable Diffusion web UI
GNU Affero General Public License v3.0
139.83k stars 26.5k forks source link

[Feature Request]: colossalai integration #4606

Open jeanhadrien opened 1 year ago

jeanhadrien commented 1 year ago

Is there an existing issue for this?

What would your feature do ?

Integrate ColossalAI improvements : https://github.com/hpcaitech/ColossalAI/tree/main/examples/images/diffusion

Proposed workflow

User is able to toggle improvements by colossal AI

Additional information

No response

Ratinod commented 1 year ago

I also want dreambooth with 8 Gb VRAM... Probably worth duplicating the Feature Request here: Dreambooth Extension for Stable-Diffusion-WebUI

sosawolfo commented 1 year ago

Integration with Colossal-AI would be amazing. The speed and VRAM improvements seem to be well worthwhile.

0xdevalias commented 1 year ago

Probably worth duplicating the Feature Request here: Dreambooth Extension for Stable-Diffusion-WebUI

Done:

Also, here are a few other alternative speed/efficiency/etc increase potential issues i'm aware of, each using different techniques/etc:

0xdevalias commented 1 year ago

It looks like the dev team of ColossalAI is looking to work directly with huggingface diffusers to simplify integrations in this issue:

This is very cool @binmakeswell ! Would be very happy to integrate diffusers in colossalAI examples!

Originally posted by @patil-suraj in https://github.com/huggingface/diffusers/issues/1212#issuecomment-1310167948

Thanks for your enthusiastic response and we look forward to working together!

How do you expect it to be carried out? Take our collaboration with Lightning for example, we communicated in slack for details, provided documentation and eventually did a joint release.

Originally posted by @binmakeswell in https://github.com/huggingface/diffusers/issues/1212#issuecomment-1311220560

And there seems to be at least some initial/cursory interest on the issue I opened on the ColossalAI repo RE: integrating with dreambooth:

Hi, thanks for reaching out! I think it’s pretty awesome. Can I check it with our team and get back to you ASAP?

Originally posted by @Sze-qq in https://github.com/hpcaitech/ColossalAI/issues/1935#issuecomment-1313053594

So hopefully things rapidly move towards the 'easier to integrate'/'better documentation' direction of things!

Originally posted by @0xdevalias in https://github.com/ShivamShrirao/diffusers/issues/129#issuecomment-1313059637

Kuri-su commented 1 year ago

any progress?

binmakeswell commented 1 year ago

Sorry for the late reply. The Colossal-AI team is busy with open source ChatGPT recently and received a lot of cooperation interest.

We are expected to have manpower to assist in this work in March. We also look forward to and welcome the participation of the open source community. Thanks.