[Bug]: Mac M2 - ANE compilation failed - Help Wanted

[trangvu9]

Is there an existing issue for this?

• [X] I have searched the existing issues and checked the recent builds/commits of both this extension and the webui

Have you read FAQ on README?

• [X] I have updated WebUI and this extension to the latest version

What happened?

When enabling AnnimateDiff, this error shows:

2023-10-19 20:51:09,976 - AnimateDiff - INFO - Hacking ControlNet.
  0%|                                                                        | 0/20 [00:00<?, ?it/s]loc("mps_add"("(mpsFileLoc): /AppleInternal/Library/BuildRoots/75428952-3aa4-11ee-8b65-46d450270006/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Core/Files/MPSGraphUtilities.mm":233:0)): error: 'anec.add' op Invalid configuration for the following reasons: The dim (=8192) at axis N exceeds the upper_bound 4096
2023-10-19 20:51:19.032 python3.10[28819:5936499]
Error:

 Error Domain=com.apple.appleneuralengine.compiler Code=1 "_ANECompiler : ANECCompile() FAILED" UserInfo={NSLocalizedDescription=_ANECompiler : ANECCompile() FAILED, NSUnderlyingError=0x600002ab5e60 {Error Domain=com.apple.appleneuralengine.compiler Code=22 "ANECCompile(/var/folders/pw/g267p7xd3xd1mp935xhrwnsw0000gp/T/com.apple.MetalPerformanceShadersGraph/mpsgraph-28819-2023-10-19_20_51_19-2764047602/) FAILED: err=(
    "Compiler internal error: The dim (=8192) at axis N exceeds the upper_bound 4096\n- From BROADCAST Layer: \"\n__OP___OP___anec.matmul_4279889_0_ne_matmul_fuse_xfm__arg1_1\"\n- Layer details:\n\n- Coming from input layers: \n{ None }\n"
)" UserInfo={NSLocalizedDescription=ANECCompile(/var/folders/pw/g267p7xd3xd1mp935xhrwnsw0000gp/T/com.apple.MetalPerformanceShadersGraph/mpsgraph-28819-2023-10-19_20_51_19-2764047602/) FAILED: err=(
    "Compiler internal error: The dim (=8192) at axis N exceeds the upper_bound 4096\n- From BROADCAST Layer: \"\n__OP___OP___anec.matmul_4279889_0_ne_matmul_fuse_xfm__arg1_1\"\n- Layer details:\n\n- Coming from input layers: \n{ None }\n"
)}}}
2023-10-19 20:51:19.032 python3.10[28819:5936499] ANE compilation failed!
2023-10-19 20:51:19.032 python3.10[28819:5936499] Issues were found in compilation, falling back to GPU

Happens in Mac M2. Python 3.10

Steps to reproduce the problem

1. Enable AnnimateDiff
2. Press generate
3. Error shown

What should have happened?

image should be generated and no error shown.

Commit where the problem happens

webui: 1.6.0 extension:

What browsers do you use to access the UI ?

Google Chrome

Command Line Arguments

No

Console logs

2023-10-19 20:51:09,976 - AnimateDiff - INFO - Hacking ControlNet.
  0%|                                                                        | 0/20 [00:00<?, ?it/s]loc("mps_add"("(mpsFileLoc): /AppleInternal/Library/BuildRoots/75428952-3aa4-11ee-8b65-46d450270006/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Core/Files/MPSGraphUtilities.mm":233:0)): error: 'anec.add' op Invalid configuration for the following reasons: The dim (=8192) at axis N exceeds the upper_bound 4096
2023-10-19 20:51:19.032 python3.10[28819:5936499]
Error:

 Error Domain=com.apple.appleneuralengine.compiler Code=1 "_ANECompiler : ANECCompile() FAILED" UserInfo={NSLocalizedDescription=_ANECompiler : ANECCompile() FAILED, NSUnderlyingError=0x600002ab5e60 {Error Domain=com.apple.appleneuralengine.compiler Code=22 "ANECCompile(/var/folders/pw/g267p7xd3xd1mp935xhrwnsw0000gp/T/com.apple.MetalPerformanceShadersGraph/mpsgraph-28819-2023-10-19_20_51_19-2764047602/) FAILED: err=(
    "Compiler internal error: The dim (=8192) at axis N exceeds the upper_bound 4096\n- From BROADCAST Layer: \"\n__OP___OP___anec.matmul_4279889_0_ne_matmul_fuse_xfm__arg1_1\"\n- Layer details:\n\n- Coming from input layers: \n{ None }\n"
)" UserInfo={NSLocalizedDescription=ANECCompile(/var/folders/pw/g267p7xd3xd1mp935xhrwnsw0000gp/T/com.apple.MetalPerformanceShadersGraph/mpsgraph-28819-2023-10-19_20_51_19-2764047602/) FAILED: err=(
    "Compiler internal error: The dim (=8192) at axis N exceeds the upper_bound 4096\n- From BROADCAST Layer: \"\n__OP___OP___anec.matmul_4279889_0_ne_matmul_fuse_xfm__arg1_1\"\n- Layer details:\n\n- Coming from input layers: \n{ None }\n"
)}}}
2023-10-19 20:51:19.032 python3.10[28819:5936499] ANE compilation failed!
2023-10-19 20:51:19.032 python3.10[28819:5936499] Issues were found in compilation, falling back to GPU

Additional information

No response

[continue-revolution]

I have no Mac so maybe someone who has Mac can help OP

[PascalJD]

I encountered the same issue. I found a temporary workaround by reducing both the Width and Height. I also disabled the stable diffusion setting "Upcast cross attention layer to float32". The generated videos now appear as moving noise. Has anyone else tried this approach, and if so, did you encounter similar results?

[Autantpourmoi]

same issue for me

[Fermiz]

same issue here on M3 pro, reducing the width/height can mitigate the issue, but encounter another issue:

*** Arguments: ('task(icby674mdg6hs6j)', 'Elon Musk is dancing', '', [], 20, 'DPM++ 2M Karras', 1, 1, 7, 64, 64, False, 0.7, 2, 'Latent', 0, 0, 0, 'Use same checkpoint', 'Use same sampler', '', '', [], <gradio.routes.Request object at 0x2f673ac90>, 0, False, '', 0.8, -1, False, -1, 0, 0, 0, <scripts.animatediff_ui.AnimateDiffProcess object at 0x2f6719090>, UiControlNetUnit(enabled=False, module='none', model='None', weight=1, image=None, resize_mode='Crop and Resize', low_vram=False, processor_res=-1, threshold_a=-1, threshold_b=-1, guidance_start=0, guidance_end=1, pixel_perfect=False, control_mode='Balanced', save_detected_map=True), UiControlNetUnit(enabled=False, module='none', model='None', weight=1, image=None, resize_mode='Crop and Resize', low_vram=False, processor_res=-1, threshold_a=-1, threshold_b=-1, guidance_start=0, guidance_end=1, pixel_perfect=False, control_mode='Balanced', save_detected_map=True), UiControlNetUnit(enabled=False, module='none', model='None', weight=1, image=None, resize_mode='Crop and Resize', low_vram=False, processor_res=-1, threshold_a=-1, threshold_b=-1, guidance_start=0, guidance_end=1, pixel_perfect=False, control_mode='Balanced', save_detected_map=True), False, False, 'positive', 'comma', 0, False, False, '', 1, '', [], 0, '', [], 0, '', [], True, False, False, False, 0, False, None, None, False, None, None, False, None, None, False, 50) {}
    Traceback (most recent call last):
      File "/Users/shuangqili/Documents/apps/stable-diffusion-webui/modules/call_queue.py", line 57, in f
        res = list(func(*args, **kwargs))
                   ^^^^^^^^^^^^^^^^^^^^^
      File "/Users/shuangqili/Documents/apps/stable-diffusion-webui/modules/call_queue.py", line 36, in f
        res = func(*args, **kwargs)
              ^^^^^^^^^^^^^^^^^^^^^
      File "/Users/shuangqili/Documents/apps/stable-diffusion-webui/modules/txt2img.py", line 55, in txt2img
        processed = processing.process_images(p)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/Users/shuangqili/Documents/apps/stable-diffusion-webui/modules/processing.py", line 732, in process_images
        res = process_images_inner(p)
              ^^^^^^^^^^^^^^^^^^^^^^^
      File "/Users/shuangqili/Documents/apps/stable-diffusion-webui/extensions/sd-webui-animatediff/scripts/animatediff_cn.py", line 118, in hacked_processing_process_images_hijack
        return getattr(processing, '__controlnet_original_process_images_inner')(p, *args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/Users/shuangqili/Documents/apps/stable-diffusion-webui/modules/processing.py", line 875, in process_images_inner
        x_samples_ddim = decode_latent_batch(p.sd_model, samples_ddim, target_device=devices.cpu, check_for_nans=True)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/Users/shuangqili/Documents/apps/stable-diffusion-webui/modules/processing.py", line 601, in decode_latent_batch
        raise e
      File "/Users/shuangqili/Documents/apps/stable-diffusion-webui/modules/processing.py", line 598, in decode_latent_batch
        devices.test_for_nans(sample, "vae")
      File "/Users/shuangqili/Documents/apps/stable-diffusion-webui/modules/devices.py", line 136, in test_for_nans
        raise NansException(message)
    modules.devices.NansException: A tensor with all NaNs was produced in VAE. Use --disable-nan-check commandline argument to disable this check.

[GeneralShan]

Same issue.

[Japskua]

Same issue here.

[cctl01]

I have this problem as well.

[ssfeather]

I've installed torch_nightly, and now I can generate animations. I installed torch_nightly by modifying webui-user.sh. You can refer to the content of webui-user.sh at https://github.com/AUTOMATIC1111/stable-diffusion-webui/discussions/7453.

[junxinzhang]

Same issue here.

[dengfeng0729]

same issue!