txt2img

[lepriko]

Hello, I would be really happy if someone can tell me what I'm missing. So If I go to the txt2img tab in automatic-1111 and generate an image using Controlnet depth with "preprocessor: depth_zoe, model: control_v11f1p_sd15_depth everything is working great. Even if I take the depth image generated by it and try to generate again using the only model: control_v11f1p_sd15_depth without preprocessor everything is working as expected. But, if I try to generate a depth map using this extension (no matter the model, I tried with every possible) and then take the depth map output to txt2img controlNet with model: control_v11f1p_sd15_depth no preprocessor. It is just not working. It generates random images nothing like the depth map. So I'm sure, I'm missing something. Can depth maps generated using this extension can be used in controlNet with control_v11f1p_sd15_depth model or should I download another model or it's impossible? I'm confused. :) So, please can someone shed some light on the topic?

[thygate]

This should work, it was working at some point anyway .. Are you using 16bit single channel PNG's ?

[lepriko]

Could you please, tell me how to check this? I can't find it in the settings. The other strange thing is that if I put the depth mask and hit preview, the preview goes blank white page. And if I put the working depth mask generated from the preprocessor it comes alright on preview.

[thygate]

On windows, right click, properties, details tab : bit depth.. For other platforms use something imagemagick's identify command..

What file type / bitdepth does the controlnet depth preview create ?

[lepriko]

Well, it says 16, it must be something else.

[thygate]

I just tested depth maps created with controlnet, and they are 24bit RGB ?! Which would be terrible since that only uses 8 bits to store the depth info.

You can try converting one generated by this extension (16bit single channel) to RGB with 8 bit per channel and see if controlnet behaves properly then ..

This is an issue with controlnet imo.

[lepriko]

Thank you, I will try that but the strange thing is that from today I got out of memory error using 24GB VRAM card. Dunno what is going on :/

[lepriko]

Just tested your approach and it is working. Thank you so much for this :) What I'm thinking is there is a way to save the generated images as 8-bit directly from the script? I'm gonna check the code, any help there will be appreciated. :)

[thygate]

If you save as jpeg it will always be rgb with 8 bits per channel since it doesn't support single channel images. jpeg is a lossy compression format, so keep that in mind too.

I would not recommend using 8 bit depth maps. Using an 8 bit depth map to generate the inpainted mesh for instance will result in banding due to the low dynamic range..

I assume controlnet is just always dividing by 256 instead of looking at the actual bit depth when loading a depthmap, so the issue is really over there imho.

[semjon00]

Won't fix, also see #291