how to save valid dng from nparray?

[vladmandic]

if i already have correct hdr image in np.ndarray, how do i save it to dng file? assuming shape=h, w, 3; dtype=uint16

i've tried all different combos i can think of and tested using:

• adobe's dng_validate
• adobe photoshop
• windows photos

tested:

- BitsPerSample=16          PhotometricInterpretation=Linear_Raw        SamplesPerPixel=1  
  vaidation pass, ps unprocessed monochrome, windows fails  
- BitsPerSample=16          PhotometricInterpretation=Linear_Raw        SamplesPerPixel=3  
  validation fail: invalid SamplesPerPixel, ps fails, windows overexposes  
- BitsPerSample=16,16,16    PhotometricInterpretation=Linear_Raw        SamplesPerPixel=3  
  validation fail: invalid SamplesPerPixel, ps fails, windows overexposes  
- BitsPerSample=16,16,16    PhotometricInterpretation=Linear_Raw        SamplesPerPixel=1  
  validation fail: invalid SamplesPerPixel, ps fails windows fails  
- BitsPerSample=16,16,16    PhotometricInterpretation=RGB               SamplesPerPixel=1  
  validation fail: PhotometricInterpretation requires NewSubFileType, ps fails, windows fails  
- BitsPerSample=16          PhotometricInterpretation=RGB               SamplesPerPixel=3  
  validation fail: PhotometricInterpretation requires NewSubFileType, ps fails, windows pass  

as far as i understand, PhotometricInterpretation.RGB should only be used for previews which are in second block, thus the error. but i cannot figure out Linear_Raw format?

thanks for the assistance!

[sschmaus]

Assuming you are working with already debayered raw sensor data strored in an ndarray with shape=h, w, 3; dtype=uint16, the correct tags would be BitsPerSample=16,16,16 PhotometricInterpretation=Linear_Raw SamplesPerPixel=3

There are a few possibilities why your code might not create dng files that are supported by Adobe software and this is very difficult to debug since Adobe doesn't give you good debug information.

The first issue might be due to a31edac85985f0fea02c9835b66dd58e4b2c6e32, I believe this change broke compatibility with Adobe software so I forked this repo for my work and reverted the change there: https://github.com/sschmaus/PiDNG/tree/AdobeCompatibility

Another pitfall may be that the Adobe DNG specification is very strict about which tags are needed if you are working with actual RAW data (usually stored with PhotometricInterpretation=Linear_Raw or PhotometricInterpretation=Color_Filter_Array)

Since you didn't mention it, make sure you have at least the ColorMatrix1tag defined, otherwise Adobe software will refuse to read your file, while Windows Photo view which is not as strict in enforcing the correct tags may still display an image (though often with very mangled colors)

This tag defines the color transformation from the raw colors that your camera captures to the XYZ color space. It's unique for every camera (model), but if you don't know it four your camera, you could use this one for debugging purposes: https://github.com/schoolpost/PiDNG/blob/66f3e16cf235443e4255a0ad382aa9d27bd1c095/examples/raw2dng.py#L21-L24

[vladmandic]

using BitsPerSample=16,16,16 PhotometricInterpretation=Linear_Raw SamplesPerPixel=3 fails dng_validate

and i would i expect as such as based on my understanding, it should be either 16 x 3 or [16,16,16] x 1

and ccm is used/required/needed only if PhotometricInterpretation=Color_Filter_Array, its not a valid tag when using Linear_Raw - that is whole point of linear_raw.

[sschmaus]

BitsPerSample should be set to [16,16,16] in your case.

As per the DNG specficication (p. 32):

The ColorMatrix1 tag is required for all non-monochrome DNG files.

So ColorMatrix1 is definitely needed if you are using an RGB image with PhotometricInterpretation=Linear_Raw.

You don't need it with PhotometricInterpretation=RGB, because that assumes your image is already mapped from the camera`s native color space to sRGB, but as you note this is only intended to be used for previews.

Just to make sure, is your image still in the camera`s native color space or is it already mapped to sRGB or any other color space?

[vladmandic]

ahhh....seems like i'll have to create a neutral colormatrix then somehow.

image is not from camera at all - its a synthetically generated hdr image, i just want to save it to dng. so yes, its 100% rgb using uint16 range.