search

apple / ml-stable-diffusion

Stable Diffusion with Core ML on Apple Silicon
MIT License
16.96k stars 950 forks source link

Crashed, when I run the ControlNet model on iPhone 14 #171

Open jiangdi0924 opened 1 year ago

jiangdi0924 commented 1 year ago

Thank you to all the contributors for their contributions that have allowed me to experience such an interesting package. I have developed an application that can run ml-stable-diffusion smoothly on iPhone14. However, I encountered some issues when trying to incorporate ControlNet features.

Problem description

When I run ControlNet on an actual iPhone 14 to generate images, it crashes due to excessive memory usage (it works fine on a simulator). Is there might be any incorrect settings when I converted the model?

python -m python_coreml_stable_diffusion.torch2coreml --convert-controlnet lllyasviel/sd-controlnet-scribble --unet-support-controlnet --convert-unet --convert-text-encoder --convert-vae-decoder --convert-vae-encoder -o sdv1_5_chunked_controlnet --model-version runwayml/stable-diffusion-v1-5 --bundle-resources-for-swift-cli --chunk-unet --attention-implementation SPLIT_EINSUM

Model

image

image

image

SaladDays831 commented 1 year ago

Hi @jiangdi0924 :) Were you able to resolve this? If so - what model/conversion did you use?

jiangdi0924 commented 1 year ago

Hi @jiangdi0924 :) Were you able to resolve this? If so - what model/conversion did you use?

Not yet 🥶， I will test the new features (quantize-nbits) of ml-stable-difusion 1.0.0

SaladDays831 commented 1 year ago

Ahhh, was hoping that you found a solution 😁 If you have any updates - pls ping!

We also have some discussions here that might help you