11 bit raw file support

[adjarar]

Hi, Thanks for making this library available! I was wondering if you could add support for 11 bit files. I have many such photos that I want to process. I looked at the source to see if I could add it myself but I don't fully understand what is going on in the packing functions.

If it's hard or you don't have the time could you point me to a resource so I can do it myself?

[schoolpost]

Hi, Thanks for making this library available! I was wondering if you could add support for 11 bit files. I have many such photos that I want to process. I looked at the source to see if I could add it myself but I don't fully understand what is going on in the packing functions.

If it's hard or you don't have the time could you point me to a resource so I can do it myself?

11-bit isn't a very common packing for image data, can you link to a sample frame of this image?

If the image is already packed "contiguously" then there is actually very little to do, just need to mark the DNG as 11-bit.

[adjarar]

When I try to save a file with PlanarConfiguration set to 1 I get the following error. Same result as when I don't set the tag:

Traceback (most recent call last): File "/home/hal/mars_raw/mastcam_decompandv3.py", line 87, in createDng(decompand(input_img, LUT0), imageName + "_test", args.outdir) File "/home/hal/mars_raw/mastcam_decompandv3.py", line 64, in createDng RAW2DNG().convert(rawImage, tags=tags, filename=filename, path=out_file_path) File "/home/hal/miniconda3/envs/gdal/lib/python3.8/site-packages/pydng/core.py", line 500, in convert dngTemplate.ImageDataStrips.append(tile) UnboundLocalError: local variable 'tile' referenced before assignment

The images I'm trying to convert are from one of the Mars landers. They have a very diffrent workflow and format. Her is the raw numpy array of a sample image image.npy.gz. It works fine if I set it to 12-bit, but the dynamic range is then ofcourse incorrect.

[schoolpost]

When I try to save a file with PlanarConfiguration set to 1 I get the following error. Same result as when I don't set the tag:

Traceback (most recent call last): File "/home/hal/mars_raw/mastcam_decompandv3.py", line 87, in createDng(decompand(input_img, LUT0), imageName + "_test", args.outdir) File "/home/hal/mars_raw/mastcam_decompandv3.py", line 64, in createDng RAW2DNG().convert(rawImage, tags=tags, filename=filename, path=out_file_path) File "/home/hal/miniconda3/envs/gdal/lib/python3.8/site-packages/pydng/core.py", line 500, in convert dngTemplate.ImageDataStrips.append(tile) UnboundLocalError: local variable 'tile' referenced before assignment

The images I'm trying to convert are from one of the Mars landers. They have a very diffrent workflow and format. Her is the raw numpy array of a sample image image.npy.gz. It works fine if I set it to 12-bit, but the dynamic range is then ofcourse incorrect.

There is no handling for any bit depth other than what is shown here: https://github.com/schoolpost/PyDNG/blob/9c04a5bbd6b193d23068de9fbdb133d16422e3a0/src/pydng/core.py#L359-L370

Just a guess of mine, you could modify the core.py in PyDNG to to handle any bit depth like this: ( the last line with the else statement. ) I'm not sure if this is exactly a fix but worth a try.

            if bpp == 8:
                tile = rawFrame.astype('uint8').tobytes()
            elif bpp == 10:
                tile = pack10(rawFrame).tobytes()
            elif bpp == 12:
                tile = pack12(rawFrame).tobytes()
            elif bpp == 14:
                tile = pack14(rawFrame).tobytes()
            elif bpp == 16:
                tile = rawFrame.tobytes()
            else:
               tile = rawFrame.astype('uint8').tobytes()

[schoolpost]

By the way can you also share the dimensions of the image are from the sample image? @adjarar

[adjarar]

 I will try your fix. The dimensions are x =1344 y = 1200.

May 12, 2021, 10:16 by @.***:

raw numpy array of a sample image > image.npy.gz https://github.com/schoolpost/PyDNG/files/6464453/image.npy.gz> . It works fine if I set it to 12-bit, but the dynamic range is then ofcourse incorrect.

By the way can you also share the dimensions of the image are from this sample?

— You are receiving this because you authored the thread. Reply to this email directly, > view it on GitHub https://github.com/schoolpost/PyDNG/issues/42#issuecomment-839568202> , or > unsubscribe https://github.com/notifications/unsubscribe-auth/ATWICDZV4US43JV2C4TOZO3TNI2NZANCNFSM44R7E52A> .

[adjarar]

The solution does not work. The pictures come out black/corupted. You can close the issue if there is no other solution.

[schoolpost]

The solution does not work. The pictures come out black/corupted. You can close the issue if there is no other solution.

Can you send me your source for the image? like the link from the internet to the RAW image data? The numpy array you sent is already putting it into a 16-bit container.

Also can you send me the specs/model of the camera that took the image?

I don't think this is any fault of PyDNG but I'm interested to help make the output work.

[adjarar]

Thank you for your help, I appreciate it. The images are a pain to work with if you want to do it from scratch. I've included my script to make it easier but it relies on a few things. Here's what you need to do.

1. All the raw images can be found here (they have C00_XXXX at the end). You need to also download the .LBL file and put it in the same directory. Clicking the download arrow should get you both.

2. They are archived in a custom archive called .DAT. To get them out you need to download and build the dat2img utility found here. It's small so it builds fast.

3. Then you need the GDAL library to read the .IMG file dat2img just made. The python binding can be easily installed with pip install gdal. However it depends on libgdal being installed. Instructions to install can be found here. Avoid building from source if you can, because it takes a while. I recommend installing everything to a conda enviorment.

4. Then finally, when the images are sent back from Mars they are compressed by applying a look up table mapping the 11 bit raw values to 8 bit. To get the information back you need to do a reverse of this opperation, they call it decompanding (just use my script).

My script takes a directory of dat or img files applies the decompanding and outputs dng. The code is short and straight forward. I've included the manufacturer information of the sensor and also the nasa document explaining the image pipeline. They do leave alot of details out so I don't know how much help it will be. cam2dng.py.gz KAI-2020-D.PDF MSL_MMM_EDR_RDR_DPSIS.PDF

[schoolpost]

Thank you for those links, helps clarify things and how they are working.

This is the output I get after running your script:

Technically speaking nothing seems to be "faulty" per se. If the data is truly 11-bit digitization ( which is strange because when I was reading the docs you sent the decompanding table is for 12-bit to 8-bit not 11? ) then technically there is nothing wrong with use 12bpp as long as you are setting your white & black levels correctly. Also you will need to obtain the correct CCM's to get accurate color because what is included in the examples with PyDNG is unlikely suited for your application.

Basically in summary: I think actually if you use 12bpp your images will be correct you are just "wasting" 1-bit of data per pixel so your image is maybe a big larger than it needs to be. So if the reason why you want to get 11-bit support working is for that reason then it is ok, but otherwise I think your script does exactly what it needs to do, you just need to get the correct color calibration values for the DNG tags to make it look more "correct/accurate"

PyDNG also includes LJ92 lossless compression if you want to save space with each output DNG file.

[adjarar]

Thanks for looking into this. The 12/11 bit is not entierly clear to me either, but I've had confirmation from NASA to use this particular LUT. It's also posted in the manual. And yes, I still need to figure out the color matrix, yours is just a place holder.

The reason I want to get it right is that I want to apply a demosaicing and super resolution algorithm. I think the image quality wil suffer if you feed it the wrong bit size. I've already noticed some strange artifacts when using the photoshop super resolution. I'm not 100% sure that that was the cause though. But I thought adding 11bit support would be an easy way to exclude the possibility. It's also harder to get things looking right in for example photoshop, because the controls will be mapped to 12 bits.

I can always try the dng sdk if necesarry so feel free to close this issue if you can't add the support right now.

[schoolpost]

It's not something I will be looking to add to the general library as it's quite an uncommon bit-depth.

That being said, the actual packing of data to arbitrary bit depths isn't too difficult. But making the packing process fast/efficient will take some work.

Cause right now you have wasted upper bits inside a 16-bit pixel with the numpy array: { 00000111 11111111 } { 00000111 11111111 }

What you need to do is align the data contiguously so pixels will take up just the amount of space they need, this will mean some pixels span partially across some bytes. ( some bytes will include the lower bits of one pixel and the upper bits of the next. ) { 11111111 } { 11111111 } { 11111111}

Maybe easier said than done, but you could create your repack method in the PyDNG library along the ones I already have made and add that if/else clause to use that packing method when you specify 11bpp.

Etiher way, hope you get your project rolling forward and thanks for sharing the information/data.