Add an option for a processed DNG output

[ghost]

Would be great to have an option to generate a 16-bit processed LinearRaw DNG for further manual tonemapping. I briefly grepped the code and see that you are using int8 during the processing so not sure if it would be easy to rewrite the program for the stated goal. Regarding the file metadata, you would probably want to still apply the white balance and change the coefficients in the output file while not doing any color transforms and just passing those and other tags to the new DNG. These use exiftool and could be helpful: https://github.com/gluijk/dng-from-tiff/blob/main/dngmaker.bat https://github.com/antonwolf/dng_stacker/blob/master/dng_stacker.bat

[Jamy-L]

To clarify, the real image processing is by default done using 32 bits, but it can be changed to 64 bits using the DEFAULT_CUDA_FLOAT_TYPE global variable in utils.py (along with the numpy and torch variants). In theory, outputing .dng should not be a big problem, especially using the relevant tools you have shared. However, the present pipeline replaces a demosaicing step, meaning that our processed images are RGB and thus have 3 channels, contrary to a raw .dng which can only have 1 mosaicked channel. I will try to add such an option, by decimating the RGB channels before saving to .dng, as well as proposing to save to .tiff format to keep the 3 channels.

[ghost]

Thank you for the clarification

However, the present pipeline replaces a demosaicing step, meaning that our processed images are RGB and thus have 3 channels, contrary to a raw .dng which can only have 1 mosaicked channel.

Please see https://www.loc.gov/preservation/digital/formats/fdd/fdd000188.shtml#notes Linear (demosaiced) DNG is also known as LinearRaw DNG (LinearRaw is a DNG metadata tag signifying that the data is not mosaiced), so you don't need to decimate the DNG channels

[Jamy-L]

You are right, I have only read about LinearRaw yesterday. After a bit of research, LinearRaw does not seem to be very popular and the many .dng generation scripts I have tried do not support it. I will now try to save the image to 3 channel raw .tiff, then convert it using Adobe's dng converter, which does support LinearRaw

[Jamy-L]

I have pushed an update: dng images can now be saved (see the updated Readme for more). They are 16 bits LineaRaw, and should contain every useful metadata. Note that this feature is not available on the online demo, please let me know if everything is working as expected

[ghost]

Thank you, I didn't do very extensive testing but it seems to work for me. However, I had to manually enter the path to the installed Exiftool on a Linux system and the image didn't save without running DNGvalidate (so I just commented those lines 💀). I would ask to add an option to specify the Exiftool binary from the terminal, or detect it automatically if installed to the PATH. Other than that, works as expected, so I'm closing this issue

[ghost]

I did find a bug, however: if an image does not have a Noise Profile tag or it has 6 values, the program would crash

[Jamy-L]

I may look into reading the Exiftool path from PATH if I have time, but it is not a priority at the moment. Note that running DNG validate really is mandatory to obtain a working dng: although the image seems fine before running it, it is actually a tiff image disguised with a dng extension, which will probably cause issues with image processing softwares.

The algorithm also requires the noise profile to predict the noise : it is not intended to work without. If your images do not have this tag, you can still estimate the noise profile of your sensor using noise profiling methods. It may be done using Adobe's softwares or with methods such as this. If you don't feel adventurous, you can take the values of the Pixel 4 sensor for ISO 100 : alpha = 1.80710882e-4 and beta = 3.1937599182128e-6, that may not be too far from your sensor.

Line 329 of handheld_super_resolution/super_resolution.py may be modified to :

    # if 'mode' in custom_params and custom_params['mode'] == 'grey':
    #     alpha = tags['Image Tag 0xC761'].values[0][0]
    #     beta = tags['Image Tag 0xC761'].values[1][0]
    # else:
        # Averaging RGB noise values
        ## IMPORTANT NOTE : the noise model exif already are NOT for nominal ISO 100
        ## But are already scaled for the image ISO.
    #     alpha = sum([x[0] for x in tags['Image Tag 0xC761'].values[::2]])/3
    #     beta = sum([x[0] for x in tags['Image Tag 0xC761'].values[1::2]])/3
    alpha = 1.80710882e-4 * ISO / 100
    beta = 3.1937599182128e-6 * (ISO / 100)**2

The program as is should however work when the image has 6 tags (2 for each color channel), which is the standart case. If the issue is persistant, could you please provide the error stack (with level 4 verbose if possible)?