All images come out like noise

[pgib]

With your recent batch of changes (7b34f1db7f11706242db45fef23b43074b02c6d9), everything downloads and compiles flawlessly; however, any image I generate looks like fancy noise:

Doesn't seem to matter what model I use.

I notice that there are errors like these when using SD 2.0.

Step 20 / 41, avg 0.5577832517169771 variance 0.019478119297940133
2022-12-31 15:23:48.552311-0800 Gauss[84281:61333299] [espresso] ANE Batch: 1 of the async requests being waited for returned errors.
2022-12-31 15:23:48.552394-0800 Gauss[84281:61333299] [espresso] ANE Batch: Async request 3 returned error: code=5 err=Error Domain=com.apple.appleneuralengine Code=5 "processRequest:model:qos:qIndex:modelStringID:options:error:: 0x9: Program Inference overflow" UserInfo={NSLocalizedDescription=processRequest:model:qos:qIndex:modelStringID:options:error:: 0x9: Program Inference overflow}
2022-12-31 15:23:48.552416-0800 Gauss[84281:61333299] [espresso] [Espresso::overflow_error] :2
Step 21 / 41, avg 0.5576753291216764 variance 0.018545113515247902
2022-12-31 15:23:49.407311-0800 Gauss[84281:61333299] [espresso] ANE Batch: 1 of the async requests being waited for returned errors.
2022-12-31 15:23:49.407400-0800 Gauss[84281:61333299] [espresso] ANE Batch: Async request 3 returned error: code=5 err=Error Domain=com.apple.appleneuralengine Code=5 "processRequest:model:qos:qIndex:modelStringID:options:error:: 0x9: Program Inference overflow" UserInfo={NSLocalizedDescription=processRequest:model:qos:qIndex:modelStringID:options:error:: 0x9: Program Inference overflow}

I'm on an M2 MacBook Air with 24GB of RAM.

[justjake]

Hmm, very odd. I’ve noticed a lot of log messages from Apple Neural Engine that don’t seem to affect output for me.

With the latest changes, I switched from “split_einsum” to “original” implementation of attention. Perhaps M2 requires split_einsum? I’m using an M1 MacBook Pro / 64gb RAM.

Did you manage to build successfully and generate images with the previous approach? I’d be curious if you have the old models that you built manually - you could try swapping from downloaded models to the hand-built ones by replacing the symlink at compiled-models/sd1.5 with the previous sd1.5 Resources directory (the one with merges.txt, Unet1.mlmodelc, etc) in it.

You can also try changing GaussKernel.swift where we initialize the StableDiffusionPipeline to use CPU & GPU instead of the neural engine and see if that improves output.

[justjake]

Another way to try split_einsum models is to move your current compiled-models folder out of the way for safekeeping, then edit the Makefile to set ATTENTION_TYPE to split_einsum, then run make again. That will download the other type of model.

[justjake]

This line in the Makefile: https://github.com/justjake/Gauss/blob/7b34f1db7f11706242db45fef23b43074b02c6d9/Makefile#L5

[pigeon]

thanks @justjake this fixed noise problem for me.

[pgib]

Yep, that's working for me now, too. Thank you!