[bug]: memref<1x1x77x77xi1>, Program Inference error on Apple M1

[bvndls]

Is there an existing issue for this?

• [X] I have searched the existing issues

OS

macOS

GPU

mps

VRAM

7.5GB

What version did you experience this issue on?

3.3.0post3

What happened?

Note: A similar issue was found on (https://github.com/invoke-ai/InvokeAI/issues/4356), but it differs in many ways

After successfully generating a 512x512 image in the Workflow Editor using Dreamshaper, I've changed the output resolution to 1024x1024 and got this error. Changing the resolution back to 512x512, changing the model, updating to v3.4.0rc1 or reinstalling InvokeAI using install.sh didn't fix the issue.

Steps to reproduce

1. Install InvokeAI using the install.sh script

Logs:

➜  InvokeAI-Installer ./install.sh
Initializing the installer. This may take a minute - please wait...

[notice] A new release of pip is available: 23.2.1 -> 23.3
[notice] To update, run: /private/var/folders/s4/ktq1zp391_v8vlcd1dk5pylc0000gn/T/invokeai-installer-tmpej7olhkb/bin/python3.11 -m pip install --upgrade pip
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
╔═════════════════════════════════════════════════════════════════════════════════════════════════ Welcome to the InvokeAI Installer ═════════════════════════════════════════════════════════════════════════════════════════════════╗
║                                                                                                                                                                                                                                     ║
║  macOS Users!                                                                                                                                                                                                                       ║
║                                                                                                                                                                                                                                     ║
║  Please be sure you have the Xcode command-line tools installed before continuing.                                                                                                                                                  ║
║  If not, cancel with Control-C and follow the Xcode install instructions at https://www.freecodecamp.org/news/install-xcode-command-line-tools/.                                                                                    ║
║                                                                                                                                                                                                                                     ║
║                            Some of the installation steps take a long time to run. Please be patient. If the script appears to hang for more than 10 minutes, please interrupt with Control-C and retry.                            ║
║                                                                                                                                                                                                                                     ║
╚═══════════════════════════════════════════════════════════════════════════════════════════════════════════ Darwin-arm64 ════════════════════════════════════════════════════════════════════════════════════════════════════════════╝

❗ Directory /Users/bvndls/invokeai already exists ❗
🛑 (re)install in this location? [y/n] (n): y

Your platform Darwin-arm64 only supports the "cpu" driver. Proceeding with that.

Followed by Installing PyTorch 🔥 and Installing the InvokeAI Application 🎨 sections (no errors)

2. Manually configure the exact way as shown here:

3. Launch the web interface

Logs:

Please enter 1-11, Q: [1] 1
Generate images with a browser-based interface
[2023-10-20 16:44:16,819]::[InvokeAI]::INFO --> Loaded 0 modules from /Users/bvndls/invokeai/nodes
/Users/bvndls/invokeai/.venv/lib/python3.11/site-packages/torchvision/transforms/functional_tensor.py:5: UserWarning: The torchvision.transforms.functional_tensor module is deprecated in 0.15 and will be **removed in 0.17**. Please don't rely on it. You probably just need to use APIs in torchvision.transforms.functional or in torchvision.transforms.v2.functional.
  warnings.warn(
>> patchmatch.patch_match: INFO - Compiling and loading c extensions from "/Users/bvndls/invokeai/.venv/lib/python3.11/site-packages/patchmatch".
>> patchmatch.patch_match: ERROR - patchmatch failed to load or compile (Command 'make clean && make' returned non-zero exit status 2.).
>> patchmatch.patch_match: INFO - Refer to https://invoke-ai.github.io/InvokeAI/installation/060_INSTALL_PATCHMATCH/ for installation instructions.
[2023-10-20 16:44:20,021]::[InvokeAI]::INFO --> Patchmatch not loaded (nonfatal)
[2023-10-20 16:44:21,466]::[uvicorn.error]::INFO --> Started server process [13067]
[2023-10-20 16:44:21,466]::[uvicorn.error]::INFO --> Waiting for application startup.
[2023-10-20 16:44:21,466]::[InvokeAI]::INFO --> InvokeAI version 3.4.0rc1
[2023-10-20 16:44:21,466]::[InvokeAI]::INFO --> Root directory = /Users/bvndls/invokeai
[2023-10-20 16:44:21,466]::[InvokeAI]::INFO --> Using database at /Users/bvndls/invokeai/databases/invokeai.db
[2023-10-20 16:44:21,473]::[InvokeAI]::INFO --> GPU device = mps
[2023-10-20 16:44:21,476]::[InvokeAI]::INFO --> Scanning /Users/bvndls/invokeai/models for new models
[2023-10-20 16:44:21,618]::[InvokeAI]::INFO --> Scanned 9 files and directories, imported 0 models
[2023-10-20 16:44:21,620]::[InvokeAI]::INFO --> Model manager service initialized
[2023-10-20 16:44:21,635]::[InvokeAI]::INFO --> Pruned 1 finished queue items
[2023-10-20 16:44:21,641]::[InvokeAI]::INFO --> Cleaned database
[2023-10-20 16:44:21,641]::[uvicorn.error]::INFO --> Application startup complete.
[2023-10-20 16:44:21,641]::[uvicorn.error]::INFO --> Uvicorn running on http://127.0.0.1:9090 (Press CTRL+C to quit)

4. Upload Workflow.json and add to queue

Logs:

[2023-10-20 16:53:47,303]::[InvokeAI]::INFO --> Loading model /Users/bvndls/invokeai/models/.cache/ef2ecfad56d695bd9086861664706992, type sd-1:main:tokenizer
[2023-10-20 16:53:48,425]::[InvokeAI]::INFO --> Loading model /Users/bvndls/invokeai/models/.cache/ef2ecfad56d695bd9086861664706992, type sd-1:main:text_encoder
loc("cast"("(mpsFileLoc): /AppleInternal/Library/BuildRoots/75428952-3aa4-11ee-8b65-46d450270006/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Core/Files/MPSGraphUtilities.mm":745:0)): error: 'anec.gain_offset_control' op result #0 must be 4D/5D memref of 16-bit float or 8-bit signed integer or 8-bit unsigned integer values, but got 'memref<1x1x77x77xi1>'
[2023-10-20 16:53:54,598]::[InvokeAI]::INFO --> Loading model /Users/bvndls/invokeai/models/.cache/ef2ecfad56d695bd9086861664706992, type sd-1:main:unet
[2023-10-20 16:54:09,906]::[InvokeAI]::INFO --> Loading model /Users/bvndls/invokeai/models/.cache/ef2ecfad56d695bd9086861664706992, type sd-1:main:scheduler
  0%|                                                                                                                                                                                                           | 0/10 [00:00<?, ?it/s]2023-10-20 16:54:14.258 Python[13067:396710] ANE Evaluation Error = Error Domain=com.apple.appleneuralengine Code=8 "processRequest:model:qos:qIndex:modelStringID:options:error:: ANEProgramProcessRequestDirect() Failed with status=0x16 : statusType=0x9: Program Inference error" UserInfo={NSLocalizedDescription=processRequest:model:qos:qIndex:modelStringID:options:error:: ANEProgramProcessRequestDirect() Failed with status=0x16 : statusType=0x9: Program Inference error}
2023-10-20 16:54:14.258 Python[13067:396710] ANE Scheduling Error = Error Domain=com.apple.appleneuralengine Code=8 "processRequest:model:qos:qIndex:modelStringID:options:error:: ANEProgramProcessRequestDirect() Failed with status=0x16 : statusType=0x9: Program Inference error" UserInfo={NSLocalizedDescription=processRequest:model:qos:qIndex:modelStringID:options:error:: ANEProgramProcessRequestDirect() Failed with status=0x16 : statusType=0x9: Program Inference error}
2023-10-20 16:54:14.263 Python[13067:396710] ANE Evaluation Error = Error Domain=com.apple.appleneuralengine Code=8 "processRequest:model:qos:qIndex:modelStringID:options:error:: ANEProgramProcessRequestDirect() Failed with status=0x16 : statusType=0x9: Program Inference error" UserInfo={NSLocalizedDescription=processRequest:model:qos:qIndex:modelStringID:options:error:: ANEProgramProcessRequestDirect() Failed with status=0x16 : statusType=0x9: Program Inference error}

Last string repeats over and over until the webui is down Current Image Shows a black image after generating the noise

Screenshots

Additional context

The first run right after installation

the same settings - 7.5GB, mps, sliced attention and etc.

Contact Details

Telegram, pm@bvndls.com

[bvndls]

https://github.com/pytorch/pytorch/issues/104642 this may be useful

[Millu]

Could you try with an updated version of InvokeAI (InvokeAI v3.4.0rc4) and see if you still experience this issue?

[bvndls]

Could you try with an updated version of InvokeAI (InvokeAI v3.4.0rc4) and see if you still experience this issue?

Can confirm, MPS with the same settings and workflow works on 3.4.0post1!

But I'm noticing slowdowns over time (after 3-4 minutes of using the same workflow with a random seed)

[2023-11-19 15:44:12,501]::[InvokeAI]::INFO --> Loading model /Users/bvndls/invokeai/models/sd-1/main/stable-diffusion-v1-5, type sd-1:main:scheduler
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:18<00:00,  1.85s/it]

[2023-11-19 15:50:41,998]::[uvicorn.access]::INFO --> 127.0.0.1:57350 - "GET /api/v1/queue/default/status HTTP/1.1" 200
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [01:09<00:00,  3.48s/it]

My assumption is that is somehow related to the cache (I'm on a base model M1 with only 8GB) and restarting the application helps, but that's temporary Closing this issue, but please note this in the future. Thanks!