search

huggingface / diffusers

🤗 Diffusers: State-of-the-art diffusion models for image and audio generation in PyTorch and FLAX.
https://huggingface.co/docs/diffusers
Apache License 2.0
26.23k stars 5.41k forks source link

Inference speed #7622

Closed rishabh063 closed 1 month ago

rishabh063 commented 7 months ago

What is the fastest way to do inference with sdxl and sd1.5 these days without suffering on quality?

Seen a lot of 3rd party libs like openvivo , stable fast , stream diffusion , deepcache etc

But there is some quality loss or problems in inference.

Looking for inpainting specifically

tolgacangoz commented 7 months ago

Have you looked at this discussion too? Also, if you haven't encountered, there are fast sampling techniques to research about. There are trade-offs most of the time -no free lunch, no silver bullet etc.

rishabh063 commented 7 months ago

bf 16 & sdpa seem to have no tradeoff IMO ,

Is there way to compile for multiple output dimensions ?

DN6 commented 7 months ago

@rishabh063 The Discussions section would be a better place for this question.

tolgacangoz commented 3 months ago

bf 16 & sdpa seem to have no tradeoff IMO ,

Is there way to compile for multiple output dimensions ?

(A discussion was not opened, so I am posting here.)

One can determine a static maximum output shape and then mask out unused parts?

github-actions[bot] commented 2 months ago

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

a-r-r-o-w commented 1 month ago

Closing due to inactivity, and as suggestions were already provided by Tolga. Please re-open if you continue to face any diffusers-specific problems. For inference acceleration, it would be better to open a Dicussion post to brainstorm