search

CompVis / stable-diffusion

A latent text-to-image diffusion model
https://ommer-lab.com/research/latent-diffusion-models/
Other
67.87k stars 10.11k forks source link

Leverage deepspeed for significantly faster inference #180

Open LifeIsStrange opened 2 years ago

LifeIsStrange commented 2 years ago

https://github.com/microsoft/DeepSpeed Deepspeed is a state of the art library that enable out of the box many optimizations for inference. While especially good for clusters, I believe it can bring optimizations to single GPUs too.

goldfish1974 commented 1 year ago

I believe DeepSpeed is supported by HuggingFace Accelerate. (https://huggingface.co/docs/accelerate/usage_guides/fsdp and https://huggingface.co/docs/accelerate/usage_guides/deepspeed)

The same docs also talks about how you can shard the training and inference so that you can use a) less GPU resources and/or b) Multiple GPUs.

This would open up a lot more people to be able to perform training which appears to be a limiting factor as the lowest GPU VRAM I've seen training on is around 10GB (Down from 24GB).

I feel that a lot of value would be added by reducing the VRAM requirements (offset against training speed) out in the Stable Diffusion public community.

Please consider this issue as it seems the implementation should be fairly simple as it can apparently be implemented with only a few lines of code changes and a matching configuration file.