search

apple / ml-stable-diffusion

Stable Diffusion with Core ML on Apple Silicon
MIT License
16.96k stars 950 forks source link

stuck during converting #226

Open JLCAIYE opened 1 year ago

JLCAIYE commented 1 year ago

Hi, I am trying to convert a v-pred model and meet a new issue. The converting process still stuck when it going to save unet.mlpakages file. I tried many times and meet same issue. Does any one know how to fix it? I am using macbook pro m1, 8GM. Thank you.

Here is the command I use: python -m python_coreml_stable_diffusion.torch2coreml --convert-unet --convert-text-encoder --convert-vae-decoder --convert-safety-checker --model-version stabilityai/stable-diffusion-2 -o sd2new --bundle-resources-for-swift-cli --chunk-unet --attention-implementation SPLIT_EINSUM --compute-unit CPU_AND_NE

And here is where it stuck: INFO:main:Converting vae_decoder INFO:main:vae_decoder already exists at sd2new/Stable_Diffusion_version_stabilityai_stable-diffusion-2_vae_decoder.mlpackage, skipping conversion. INFO:main:Converted vae_decoder INFO:main:Converting unet /Users/felix/ml-stable-diffusion/python_coreml_stable_diffusion/torch2coreml.py:679: FutureWarning: Accessing config attribute cross_attention_dim directly via 'UNet2DConditionModel' object attribute is deprecated. Please access 'cross_attention_dim' over 'UNet2DConditionModel's config object instead, e.g. 'unet.config.cross_attention_dim'. args.text_encoder_hidden_size or pipe.unet.cross_attention_dim or pipe.text_encoder.config.hidden_size, WARNING:python_coreml_stable_diffusion.unet:use_linear_projection=True is ignored! INFO:main:Sample UNet inputs spec: {'sample': (torch.Size([2, 4, 96, 96]), torch.float32), 'timestep': (torch.Size([2]), torch.float32), 'encoder_hidden_states': (torch.Size([2, 1024, 1, 77]), torch.float32)} INFO:main:JIT tracing.. /Users/felix/ml-stable-diffusion/python_coreml_stable_diffusion/layer_norm.py:61: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! assert inputs.size(1) == self.num_channels INFO:main:Done. INFO:main:Converting unet to CoreML.. WARNING:coremltools:Tuple detected at graph output. This will be flattened in the converted model. Converting PyTorch Frontend ==> MIL Ops: 0%| | 0/10456 [00:00<?, ? ops/s]WARNING:coremltools:Saving value type of int64 into a builtin type of int32, might lose precision! Converting PyTorch Frontend ==> MIL Ops: 100%|▉| 10454/10456 [00:02<00:00, 4290. Running MIL frontend_pytorch pipeline: 100%|█| 5/5 [00:00<00:00, 10.82 passes/s] Running MIL default pipeline: 100%|████████| 64/64 [05:41<00:00, 5.34s/ passes] Running MIL backend_mlprogram pipeline: 100%|█| 11/11 [00:00<00:00, 35.08 passes

jrittvo commented 1 year ago

I just ran your command on my M1Pro 16GB. It also failed to complete successfully. Seemed to fail at the same place as yours did, with Activity Monitor showing the ANEcompiler process stuck. I rebooted to completely clear that process.

Then I tried again, this time without the last argument, the --compute-unit CPU_AND_NE. For SPLIT_EiNSUM conversion targets, I have never included a --compute-unit argument of any kind. There is a 3 or 4 minute lag after the last line you show above, where it seems like nothing further is going to happen, but it does in fact write the Stable_Diffusion_version_stabilityai_stable-diffusion-2_vae_decoder.mlpackage after that lag, and it goes on to complete the whole conversion successfully (with a few more "lags" along the way). The total process is probably around 15 minutes.

Can you try without using the --compute-unit CPU_AND_NE argument, and report back here if that worked for you?