add_colormap not filling in missing values sometimes

[ScottLindstrom]

I created two different Red.txt enhancement tables, Red0.txt

0,0,0,0,255
255,255,0,0,255

and Red1.txt

0,0,0,0,255
1,1,0,0,255
255,255,0,0,255

I expected the Red0.txt to show only different shades of Red for my greyscaled image (the case I used was C13). It didn't But Red1.txt did.

geo2grid.sh -r abi_l1b -w geotiff -p airmass wvdif C13 -f /arcdata/goes_restricted/grb/goes16/2022/2022_12_09_343/abi/L1b/RadC/*s20223430001*
add_colormap.sh ../../enhancements/Red0.txt   GOES-16_ABI_RadC_C13_20221209_000117_GOES-East.tif
add_coastlines.sh --add-coastlines --coastlines-resolution f --coastlines-level 5  --add-color *C13*1209_0001*.tif
display GOES-16_ABI_RadC_C13_20221209_000117_GOES-East.png

If you then re-run but use Red1.txt, you'll get different results.

[djhoese]

If you then re-run but use Red1.txt, you'll get different results.

You mean with the same geotiff running it over and over again?

[ScottLindstrom]

No, starting from the geo2grid.sh command -- rerun all 4 commands.

[djhoese]

So when I run this with red0 on C08 (just because I know that has a default linear scaling/enhancement) I get this:

$ add_colormap.sh red0.txt GOES-16_ABI_RadC_C08_20181125_125718_GOES-East.red0.tif
/home/davidh/repos/git/trollimage/trollimage/colorspaces.py:128: RuntimeWarning: invalid value encountered in power
  1.055 * (arr ** (1.0 / 2.4)) - 0.055,

I think this is the colorspace conversion (RGB -> HSL -> interpolation -> HSL -> RGB) having a problem starting from black. I thought this sounded familiar so I looked and it turns out I filed an issue with our trollimage package a while ago:

https://github.com/pytroll/trollimage/issues/106

CC @mraspaud

Here is what the red0 image looks like for reference:

~Now about the changing output when you re-run it @ScottLindstrom. What type of different output?~ Edit: Clarified on slack. The output between red0 and red1 is different, not that the result is always changing with each execution.

[djhoese]

So this is a bug in the trollimage package which is what is used to handle all colormap/colorspace conversions. The above mentioned issue is tracking the transition to a different package for handling these conversions. When I use this new package I get this as my image with red0:

[djhoese]

Given the timing of this we won't be able to include this in G2G 1.1.0 which should be released next week, but I think we can set a goal to create a G2G 1.1.1 with these updates and fixes. I would hope we could release 1.1.1 right after P2G 3.0 is released as P2G will need these fixes as well.

[djhoese]

This should be fixed once this trollimage PR is merged, released, and P2G/G2G told to use the new version:

https://github.com/pytroll/trollimage/pull/122