Unable to reconstruct RGB from HSI

[beatreichenbach]

Hello,

Thank you so much for sharing your work! This looks super interesting.

I'm trying to use your algorithm for a VFX implementation to do a round trip from RGB -> HSI -> RGB. I'm using the model mst_plus_plus.pth provided from Google Drive with the code provided here: https://github.com/caiyuanhao1998/MST-plus-plus/blob/master/predict_code/test.py

I'm then converting the HSI to RGB using the CIE (2006) 10-deg but also tried CIE_1964 from this repo by adding up the values for all wavelengths:

# spectral_array is the output from the prediction code.
height, width, lambda_count = spectral_array.shape
lambda_values = np.linspace(400, 700, lambda_count)
image_shape = (height, width, 3)
output = np.zeros(image_shape, np.float32)
for w in range(lambda_count):
    # rgb_dict is the ciexyz values
    rgb = rgb_dict[lambda_values[w]]
    for y in range(height):
        for x in range(width):
            output[y, x] += rgb * spectral_array[y, x, w]

I'm simply add up all the different colors from all 31 wavelengths and saving out the file after converting the CIE-XYZ-D65 to sRGB color space. Unfortunately the resulting rgb values are about 16 times brighter, and don't really match up. I attached a reconstructed rgb image with reduced intensity (output /= 16).

The colors definitely match, so the overall distribution seems pretty good, but the intensity does not seem accounted for. Is there anything to consider to be able to reconstruct an image and preserve the intensity?

[caiyuanhao1998]

Hi, naively adding up the HSI channels cannot derive the corresponding RGB image. Please refer to this repo. Boaz Arad provide the code for HSI-to-RGB conversion

[beatreichenbach]

Thank you so much for sending me in the right direction.

[caiyuanhao1998]

you are welcome. If you found our repo helpful, please help us star it. Thanks