search

ssec / polar2grid

Tools for reading, remapping, and writing satellite instrument data.
http://www.ssec.wisc.edu/software/polar2grid/
GNU General Public License v3.0
71 stars 33 forks source link

Surface Reflectance True Color Image Issues #664

Closed kathys closed 6 months ago

kathys commented 6 months ago

The P2G True Color imagery that is produced by the VIIRS EDR Surface Reflectance reader shows alot of red in clouds. Here is an example from a December 5 image segment over the Yucatan. For comparison, I have attached an SDR true color image (top) and an image of same scene created from the Surface Reflectance product (bottom). I do not have log files since this was part of the operational processing, and the P2G logs are not saved. However, I can provide cutouts with logs or whatever you might need.

VIIRS_true_color_SDR_20231205_noaa20

VIIRS_true_color_SURF_20231205_noaa20

This dataset (which is long) is located here: bumi:/data/users/kathys/test_data/viirs_edr/j01/winter

kathys commented 6 months ago

Liam and I discussed this. The problem looks to be that the I-Band 1 is the only surface reflectance band image involved in the creation of true color RGB where those cumulus clouds can be seen. So we would like to implement two things to see if this problem can be resolved:

1). Use I-Band 2 for sharpening the image instead of I-Band 1 2). Down sample I-Band 1 and use that for the Red component of the RGB true color instead of M-Band 5.

djhoese commented 6 months ago

These should be relatively easy for you to test out on your own @kathys. For number 2, you should be able to load/request true_color_lowres_surf which will use M05, M04, M03 and no sharpening. You can find the definition for that in the Satpy installation in the bundle and a copy of it is here:

https://github.com/pytroll/satpy/blob/6d652a613808993e14e6e38bd3b3fca26976b5dd/satpy/etc/composites/viirs.yaml#L337-L343

If you change the M05 to I01 (or make a copy of it with a different name) then the composite should only be made after resampling when I01, M04, and M03 are all the same size.

For 1, I'm not sure I understand. I02 is the 0.86um band right? That isn't used for the true color composite? M05 (0.67), M04 (0.555), M03 (0.488).

kathys commented 6 months ago

I did try changing the local viirs.yaml file as you suggested above. That did not improve the true color image in and around the cumulus clouds, but it did result (of course) in a reduced resolution image. I then tried replacing the M05 with I01 in the true_color_surf reader definition, and that resulted in the exact same image as was produced in the true_color_lowres_surf. I did not expect this.

djhoese commented 6 months ago

Summary of our meeting today: The I01 band makes things bad, but it isn't the whole story. @kathys generated a true color with just M05, M04, and M03 and it did not have the red pixels identified here. The non-sharpened I01, M04, M03 true color did have these red pixels. However, the low resolution image (M05/M04/M03) still had missing pixels in the lower left of the swath in question and blue "scars" on the top of the swath (most in the top left). So this is clearly in the data so the questions and choices are:

  1. Do we make true_color_surf only be the low resolution version but still warn users to not use it operationally? There could also be language about "as the algorithm improves the composite will improve".
  2. Do we keep it as is with a more concerned warning: "most of the image is good :shrug:"
  3. Are there quality flags that could help (probably not)?
  4. Other ideas?

Side note: This testing revealed a bug in the viirs_edr reader with newer versions of xarray so I'll be fixing that in Satpy.

kathys commented 6 months ago

I had a conversation with Liam this morning. I told him about our findings with Land Surface Reflectance true color image creation. I think we agreed that at this point it would take a long time to work on making these images better. So once I do a little more research, including looking at NOAA CLASS LSR products, then we will be done. I will just document the artifacts that are in the products.

kathys commented 6 months ago

For completeness, here is a comparison between the NOAA LSR images and the CSPP LSR true color images from 5 December 2023 over a couple of different places. The CSPP images show a more red hue to the images. There are still red clouds over the Yucatan Peninsula and surrounding waters, but not as many. NOAA algolrithm is version v1r3 and CSPP released v1r2. So the LSR products will be improved with future CSPP releases, and in turn, the true color images too. Should note that the other artifacts noted by Dave above, including the blue scars, are still in the NOAA data in v1r3. Top pics are the images made from NOAA LSR, bottom pics are made from CSPP LSR.

NOAA_LSR_true_color_Plains CSPP_LSR_true_color_Plains CSPP_LSR_true_color_Yucatan NOAA_LSR_true_color_Yucatan

djhoese commented 6 months ago

Do we close this as there isn't anything this project needs to do?

kathys commented 6 months ago

Yes!

kathys commented 6 months ago

There are a number of issues with the true color images created from the CSPP Surface Reflectance product. To fix them would take a lot of work, since these issues are a result of problems with the input data. Some of these issues will be resolved once the CSPP Land Surface Reflectance product is updated to version 1.3. We should get that software in April. Closing this issue.