How to resolve 16 bit depth map issues - Photoshop warning

[AugmentedRealityCat]

TLDR: Photoshop saves 16 bit PNG images as 15 bit instead, and this doesn't work well. Solution: use another app, like Krita.

If your custom depth map input is a standard PNG or JPG in 8 bit per channel RGB (aka 24 bit, or 32 bit if there is an alpha) then everything should work properly. So that's OK.

If you use a depth map that has been generated by the High Resolution Depth Map script, it will be a PNG in 16 bit, single channel greyscale, and this works even better. It's very useful as it lets you use different algorithms to extract depth maps, and depending on the scene they might do a better job than the 2.0 depth model does by itself. So that's OK as well.

BUT If you use a depth map that has been saved from Photoshop (and from many other applications like Cinema4d for example), then you will run into problems. You will load it and it will show up in the little window and look all nice. And then you'll press generate and get some kind of very flat 2d silhouette of your depth map. Since the Depth-Image-IO script from this repo outputs the depth map you just used as well as the image it has just generated, you can check that depth image output to see how your own depth image was interpreted. And if it was a 16 bit PNG from Photoshop, then 100% of the time it will come out flat, and the resulting depth image will be a flat white silhouette over a black background, and no other depth detail whatsoever.

If you check the format of that PNG from Photoshop and compare it with a PNG produced with the High Resolution Depth Map Script extension they will apparently be identical. But if you take one Depth Map that works well (from either the Depth Image IO script over here, or the High Resolution Depth Map script) and load it in Photoshop and save it back as a PNG 16 bit greyscale from there - so the exact same format (supposedly) - then it will behave just as badly as any 16 bit PNG depth image saved from Photoshop.

But why ? Because Photoshop for some strange legacy reasons interprets 16 bit greyscale PNG as 15 bits +1. This truncation of the initial range implies a necessary loss of data and is the root of our problems when they get used as inputs for the Depth-Image-IO script we are discussing over here.

Here is more information about the problem - we are not the first, and clearly not the last to suffer from Adobe's decision to do this, and furthermore to do it without documenting it officially anywhere. Anyways... https://community.adobe.com/t5/photoshop-ecosystem-discussions/export-as-png-at-16-bits-does-not-work-in-ps-version-23-2-1/m-p/12813087 https://community.adobe.com/t5/photoshop-ecosystem-discussions/question-about-16bit-images-in-photoshop/m-p/10050694

So, what is the solution ?

SOLUTION: use another application than Photoshop to save your PNG 16 bit greyscale Depth Images for use as inputs for the Depth-Image IO script from this repo. Which application ? I have tested Krita, and it does the job very well. You can still edit your depthmap in Photoshop if you want, just save it in a different 16 bit format instead of PNG (like TIF for example) and then import that image in Krita or any other software that would have been proven to work properly with those insidious 16 bit PNGs.

WHAT DOES NOT WORK: Photoshop 16 bit PNG, Cinema4d 16 bit PNG

Would it be useful to build a list of softwares known to work well for exporting full 16 bit greyscale PNGs ?

[AnonymousCervine]

By any chance can you hand me a couple sample/example files of the sort that are breaking? (I'm not a professional artist and don't own photoshop—and anyway it would be easier on me to start from a specific file known-not-to-be-working.)

(If you're feeling extra nice I'd be happy to have one of the 16-bit images that do work so I don't have to rustle one up.)

I only tested the current code on 8-bit images in this first release, and I'm not at all surprised it's breaking on various other formats—however 15-bit precision vs 16-bit precision causing trouble is actually somewhat surprising to me. It may be that gradio or webui mangle something before it gets to the extension, but I can check.

[AugmentedRealityCat]

By any chance can you hand me a couple sample/example files of the sort that are breaking?

I will do that for you later today, no problem.

[rockerBOO]

For GIMP, setting the Pixel format as 16bit RGBA or GRAYA works, but the others don't seem to. Seems to need the alpha? I made the depth map from the High Resolution Depth map script, and it seems to be 8 bit grayscale.

Here is one from the High Resolution Depth Map script (that is 8 bit depth):

[AnonymousCervine]

Firstly, thanks to everyone for reporting on the state of things!

I've pushed a fix to such 16-bit image types as were having difficulty that I could identify (which may not even be everything already mentioned in this thread—images from Photoshop for instance remains untested, though it sounds potentially like the same issue). I'll leave this issue open for at least a couple of days to see if anyone reports formats that aren't working—and if you do, please provide the/an image file that isn't working too! Such things help keep me sane.

For any curious what was going on:

To be on the safe side with formats, I had told PIL (that is to say, the Python Imaging Library, or rather its fork Pillow) to convert every image to RGB format so that strange formats (say, a palletized GIF, or a JPEG in YCbCr) could all be treated identically without any surprises. (Incidentally, Gradio appears to have made similar assumptions and makes the same conversion under the hood unless I tell it not to, which thankfully I can). RGB might seem like an odd choice for greyscale images but it seemed like it would be the 'safer', more battle-tested option compared to single-channel.

Unfortunately my small bit of caution backfired: It seems PIL does not always do a very good job at the conversion. Some 16-bit images just get clamped from their 16-bit values (0 to 65535) to 8-bit values (0 to 255)—resulting in everything but the darkest blacks turning to pure white, which depth2image of course just interprets as a flat plane.

So, now I only convert the image to RGB if it's in a format that obviously would cause problems (for example CYMK, though who would do that), and otherwise just blindly take the first image channel and hope it works. This, ironically, seems to have much better results in practice, for all that it makes me a little uncomfortable—it's at least no worse than whatever PIL was doing.

[AugmentedRealityCat]

I am going to retest everything with the new code and I'll let you know how it goes.

[AnonymousCervine]

Much appreciated!

[AugmentedRealityCat]

With the latest version all the 16 bit versions are working very well including those from photoshop. This is the conclusion after a whole series of tests I just completed.

I think we can close this warning about photoshop files as it's mostly irrelevant for this script now.

Thanks a lot for solving this little problem ! Much appreciated.