search

brkirch / stable-diffusion-webui

Stable Diffusion web UI
GNU Affero General Public License v3.0
195 stars 10 forks source link

[Feature Request]: new attention optimization #39

Open Nidal890 opened 11 months ago

Nidal890 commented 11 months ago

Is there an existing issue for this?

What would your feature do ?

Wondering if it is possible to incorporate metal-flash-attention into this project. Webui currently uses the mps backend, meanwhile metal flash attention is an open source alternative to mps based off of dao AI labs flashattention v2. It's mainly built for apple silicon gpus and is much faster and less resource hungry than mps. Not sure about amd radeon gpus on macos though. I used it on the draw things app and it's really good. I know that it can't do fp64 calculations and some other stuff I don't understand but I though I should share. The linked repo has more information and benchmarks. https://github.com/philipturner/metal-flash-attention This article article talks more in detail about mfa: https://engineering.drawthings.ai/integrating-metal-flashattention-accelerating-the-heart-of-image-generation-in-the-apple-ecosystem-16a86142eb18

Proposed workflow

  1. Go to ....
  2. Press ....
  3. ...

Additional information

No response

tiancool commented 10 months ago

Interrogate CLIP now runs on MPS instead of CPU, how did you do it? This error occurred in my original version, how should I fix it? Thank you for your contribution.

Nidal890 commented 10 months ago

@tiancool Unfortunately, only have the most basic of python skills and can't add it myself otherwise I'd this would be a PR. I've been paying attention to the work in this repo and thought I'd at least share the MFA repo I found along with info.

tiancool commented 10 months ago

@tiancool Unfortunately, only have the most basic of python skills and can't add it myself otherwise I'd this would be a PR. I've been paying attention to the work in this repo and thought I'd at least share the MFA repo I found along with info.

Tested it for a few days, love your modified and optimized version, very stable, thanks a lot!

Nidal890 commented 10 months ago

@tiancool Unfortunately, only have the most basic of python skills and can't add it myself otherwise I'd this would be a PR. I've been paying attention to the work in this repo and thought I'd at least share the MFA repo I found along with info.

Tested it for a few days, love your modified and optimized version, very stable, thanks a lot!

Nah, you have to thank philipturner and liuliu for putting in all the work to make mfa as good as it is. Hopefully, we can see it implemented here in this repo