Community Integration: Making AIGC cheaper, faster, and more efficient

[binmakeswell]

AIGC has recently risen to be one of the hottest topics in AI. Unfortunately, large hardware requirements and training costs are still a severe impediment to the rapid growth of the AIGC industry. The Stable Diffusion v1 version of the model requires 150,000 A100 GPU Hours for a single training session.

We are happy to share a fantastic solution where the costs of training AIGC models such as stable diffusion can be 7 times cheaper!

Colossal-AI release a complete open-source Stable Diffusion pretraining and fine-tuning solution with the pretraining cost reduced by 6.5 times, and the hardware cost of fine-tuning by 7 times. An RTX 2070/3050 PC is good enough to accomplish the fine-tuning task flow, allowing AIGC models such as Stable Diffusion to be available to a wider community.

Open-source code：https://github.com/hpcaitech/ColossalAI/tree/main/examples/images/diffusion

More details can be found on the blog. We believe the democratization of AIGC models is also very helpful for Stable Diffusion users. We would appreciate it if we could build the integration with you to benefit both of our users, and we are willing to provide help you need in this cooperation for free.

Thank you very much.

Best regards, Yongbin Li, HPC-AI Tech

[arifsaeed]

Can you explain what the diffusion_pytorch_model.bin file is, im assuming that the model ckpt file. What if i wanted to use the newly released v1.5 file how could i add it. from_pretrained: '/data/scratch/diffuser/stable-diffusion-v1-4/unet/diffusion_pytorch_model.bin'

[Fazziekey]

Our default config use the CompVis/stable-diffusion-v1-4

git lfs install git clone https://huggingface.co/CompVis/stable-diffusion-v1-4

[Fazziekey]

if you want to used v1.5 please git clone the https://huggingface.co/runwayml/stable-diffusion-v1-5/tree/main

[Fazziekey]

we add the model down load in README https://github.com/hpcaitech/ColossalAI/pull/1958