Depth-map reduced to white in output

[YanWittmann]

I seem to have some trouble getting this extension to work;

I use the 512-depth-ema.ckpt with an according 512-depth-ema.yaml file, which works in img2img. When I ask the UI to generate an image on the text2img tab with the extension script active, the depth map is reduced to almost white however:

And the whole UI:

The error might very well be on my part, can someone tell me where I went wrong? Thank you!

Details:

• Python 3.10.9 (tags/v3.10.9:1dd9be6, Dec 6 2022, 20:01:21) [MSC v.1934 64 bit (AMD64)]
• latest version of the Automatic UI (226d840e84c5f306350b0681945989b86760e616)

[AnonymousCervine]

There have been reported issues with different image formats already (namely, issue #2 ); by any chance can you give me a copy of the file used for the input depth image here?

(It almost certainly isn't going to be a problem with the configuration in this case. Based on that screenshot, the extension isn't parsing the input depth image correctly.)

[YanWittmann]

Ah, the image format. All right, I got it to work

• by converting the image into a JPG
• by reexporting the image from an image editor as PNG

Here's the original PNG:

And here's the re-exported PNG version:

Here are some results for The face of a young man, highly detailed photography

I am having some trouble adding a red shirt however, do you have any suggestions on how to not make the entire face red (which is the case for almost every generation I make with this prompt)? The face of a young man, highly detailed photography wearing a red shirt

Thank you!

[AnonymousCervine]

First, thank you for the source images, they're most helpful!

Since it does seem to be the same format problem, I'm going to mark this issue as a duplicate of #2 (any updates will be posted there)

I am having some trouble adding a red shirt however, do you have any suggestions on how to not make the entire face red

I too have noticed that the depth2img model is honestly a little bit wonky about prompts sometimes compared to normal SD models we're used to—perhaps especially with regard to colours.

Depending on your goals: You can always do it in two steps and use inpaint! There is a bug where SD doesn't handle inpainting correctly for depth2img but if you supply the depth manually with this extension (i.e. the same depth image, again) it will respect it. (Though it will still may have that certain "I am inpainted" look; on the other hand, it doesn't get the general structure of things wrong as much as normal inpainting because it has the depth to guide it.)

So for instance, using your first prompt and getting this (emulated stock-photo bands and all! And strikingly similar to the image you got, really. I do wonder if in general the structure provided by the depth half of depth2img model-input doesn't make SD a little less creative sometimes):

Masked like so:

With prompt "a young man wearing a (red:1.2) shirt" (emphasis only on the things inpainting needs to know, and you'll notice I had trouble getting it to pick up the idea of 'red' again in an image without it); cherry-picked best result out of 12-ish:

(Note that it's still following the shoulder contours suggested by the depth image. Otherwise, it's just a normal-ish inpainting job.)