search

JCSDA / CRTMv3

CRTMv3 repository for coordinated development and releases. Code history is not carried in this repository prior to v3, to reduce the cloning overhead. For v2.x history leading up to v3, see JCSDA/crtm repository.
Other
6 stars 6 forks source link

Initial implementation of TOA reflectance is quirky #95

Closed gthompsnJCSDA closed 6 months ago

gthompsnJCSDA commented 10 months ago

Related issues to this topic:

69

84

and UFO Issue 3135

I created a single observation point from GOES-16 (ABI) data at 1800 UTC 15 Feb 2022 at (lat,lon) = (-8.3219, -78.3487) with the following values of observed reflectance at channels 1-6 respectively: 0.9070639, 0.8194615, 0.8824381, 0.245723, 0.216781, 0.2842982. The point and time were picked due to being near solar subpoint to eliminate issue of cos(solarZenithAngle) and observing a deep cloud. The GFS-FV3 data in GeoVaLs also has ice cloud aloft followed by snow and cloud water in the column.

A print statement was placed in src/RTSolution/Common_RTSolution.f90 in the branch feature/cd_TOA_R where I also added a test to ensure that Solar_irradiance was greater than ONE.

    ! -------------------------------------------------------
    ! Compute the corresponding solar reflectance (TOA only)
    ! -------------------------------------------------------
    IF( RTV%Solar_Flag_true .AND. (.NOT.(RTV%aircraft%rt))) THEN
      RTSolution%Reflectance = ZERO
      IF(RTV%Solar_irradiance .gt. ONE) THEN
         RTSolution%Reflectance = RTSolution%Radiance/(RTV%COS_SUN*RTV%Solar_irradiance/PI)
         write(*, '(a,f9.4,a,f9.4,a,f7.4)') ' DEBUG-GREG, Solar_irradiance, Down_Solar, reflectance = ', &
                        RTV%Solar_irradiance, ', ', RTV%Down_Solar_Radiance, ', ', RTSolution%Reflectance
      END IF
    END IF

This results in a series of 7 lines of print statements per 6 channels as per the following:

 DEBUG-GREG, Solar_irradiance, Down_Solar, reflectance =   46.8226,    0.0000,  0.4462
 DEBUG-GREG, Solar_irradiance, Down_Solar, reflectance =   46.8226,   12.8284,  0.0308
 DEBUG-GREG, Solar_irradiance, Down_Solar, reflectance =   46.8226,    0.0000,  0.4339
 DEBUG-GREG, Solar_irradiance, Down_Solar, reflectance =   46.8226,    0.0000,  0.4337
 DEBUG-GREG, Solar_irradiance, Down_Solar, reflectance =   46.8226,    0.0000,  0.4337
 DEBUG-GREG, Solar_irradiance, Down_Solar, reflectance =   46.8226,    0.0000,  0.4337
 DEBUG-GREG, Solar_irradiance, Down_Solar, reflectance =   46.8226,    0.0000,  0.4337
 DEBUG-GREG, Solar_irradiance, Down_Solar, reflectance =   68.4788,    0.0000,  0.4204
 DEBUG-GREG, Solar_irradiance, Down_Solar, reflectance =   68.4788,   20.0645,  0.0970
 DEBUG-GREG, Solar_irradiance, Down_Solar, reflectance =   68.4788,    0.0000,  0.4059
 DEBUG-GREG, Solar_irradiance, Down_Solar, reflectance =   68.4788,    0.0000,  0.4056
 DEBUG-GREG, Solar_irradiance, Down_Solar, reflectance =   68.4788,    0.0000,  0.4056
 DEBUG-GREG, Solar_irradiance, Down_Solar, reflectance =   68.4788,    0.0000,  0.4056
 DEBUG-GREG, Solar_irradiance, Down_Solar, reflectance =   68.4788,    0.0000,  0.4056
 DEBUG-GREG, Solar_irradiance, Down_Solar, reflectance =   73.8129,    0.0000,  0.4823
 DEBUG-GREG, Solar_irradiance, Down_Solar, reflectance =   73.8129,   22.7923,  0.2290
 DEBUG-GREG, Solar_irradiance, Down_Solar, reflectance =   73.8129,    0.0000,  0.4669
 DEBUG-GREG, Solar_irradiance, Down_Solar, reflectance =   73.8129,    0.0000,  0.4666
 DEBUG-GREG, Solar_irradiance, Down_Solar, reflectance =   73.8129,    0.0000,  0.4666
 DEBUG-GREG, Solar_irradiance, Down_Solar, reflectance =   73.8129,    0.0000,  0.4666
 DEBUG-GREG, Solar_irradiance, Down_Solar, reflectance =   73.8129,    0.0000,  0.4666
 DEBUG-GREG, Solar_irradiance, Down_Solar, reflectance =   70.4598,    0.0000,  0.0060
 DEBUG-GREG, Solar_irradiance, Down_Solar, reflectance =   70.4598,    0.0470,  0.0000
 DEBUG-GREG, Solar_irradiance, Down_Solar, reflectance =   70.4598,    0.0000,  0.0065
 DEBUG-GREG, Solar_irradiance, Down_Solar, reflectance =   70.4598,    0.0000,  0.0065
 DEBUG-GREG, Solar_irradiance, Down_Solar, reflectance =   70.4598,    0.0000,  0.0065
 DEBUG-GREG, Solar_irradiance, Down_Solar, reflectance =   70.4598,    0.0000,  0.0065
 DEBUG-GREG, Solar_irradiance, Down_Solar, reflectance =   70.4598,    0.0000,  0.0065
 DEBUG-GREG, Solar_irradiance, Down_Solar, reflectance =   65.0004,    0.0000,  0.4789
 DEBUG-GREG, Solar_irradiance, Down_Solar, reflectance =   65.0004,   19.9822,  0.1957
 DEBUG-GREG, Solar_irradiance, Down_Solar, reflectance =   65.0004,    0.0000,  0.4671
 DEBUG-GREG, Solar_irradiance, Down_Solar, reflectance =   65.0004,    0.0000,  0.4669
 DEBUG-GREG, Solar_irradiance, Down_Solar, reflectance =   65.0004,    0.0000,  0.4669
 DEBUG-GREG, Solar_irradiance, Down_Solar, reflectance =   65.0004,    0.0000,  0.4669
 DEBUG-GREG, Solar_irradiance, Down_Solar, reflectance =   65.0004,    0.0000,  0.4669
 DEBUG-GREG, Solar_irradiance, Down_Solar, reflectance =   40.0316,    0.0000,  0.4354
 DEBUG-GREG, Solar_irradiance, Down_Solar, reflectance =   40.0316,   12.1132,  0.0629
 DEBUG-GREG, Solar_irradiance, Down_Solar, reflectance =   40.0316,    0.0000,  0.4257
 DEBUG-GREG, Solar_irradiance, Down_Solar, reflectance =   40.0316,    0.0000,  0.4254
 DEBUG-GREG, Solar_irradiance, Down_Solar, reflectance =   40.0316,    0.0000,  0.4254
 DEBUG-GREG, Solar_irradiance, Down_Solar, reflectance =   40.0316,    0.0000,  0.4254
 DEBUG-GREG, Solar_irradiance, Down_Solar, reflectance =   40.0316,    0.0000,  0.4254

The second line in each set is the only print with non-zero Down_Solar value.

The Solar_irradiance values do not appear correct, even considering the factor of PI, because near the solar subpoint should be over 1000W/m2.

Hopefully a single column test can narrow down what is happening and the path to a solution.

BenjaminTJohnson commented 10 months ago

@gthompsnJCSDA Digging my way through this, solar_irradiance from CRTM column 2, E_sun in Column 3 from Table 6a (GOES ABI ATBD), pg 21 in particular to see the relationship between the expected solar constant values and the Esun values. 

ch     solar_irradi, Esun      
--
1       46.8353729  44.1868715   
2       68.4577789  66.3274497
3       73.7946090  72.6721072
4       70.4625507  67.9143879
5       65.0208639  63.4454241
6       40.0745478  38.1148813

Solar_Irradiance is unchanged any changes in geometry information source zenith, azimuth, sensor etc. So that's our reference quantity. Units: (mW/(m^2·cm^{-1})).

In particular, the ATBD notes that bands 1 - 6 has different radiance units than bands 7-16 (pg 20) -- and the equation is what's necessary to convert. The GOES data probably contains updated Esun values, I don't have a quick way to check this.

Next I will sort out where the solar_irradiance values are computed.

BenjaminTJohnson commented 10 months ago

Solar_Irradiance comes from the SpcCoeff files for each instrument. So we'll have to provide some correction factor for each instrument to accommodate the difference between what CRTM assumes and what the instrument files assume. Here's what's in the GOES-18 ABI instrument SpcCoeff file: Solar_Irradiance = 45.249382046603, 66.1395863631183, 71.2956948943939, 68.0764704168738, 62.8190559920795, 38.7174994906341 ; so now I'm re-confused. Tracking through the code to see if solar_irradiance gets modified somewhere, because these are different from what's being output from CRTM. Still digging.

gthompsnJCSDA commented 10 months ago

To keep the documentation going in this thread, the esun values (in units of W m-2 um-1) directly in GOES (16) L1b files for channels 1-6 are the following:

channel 1 = 2017.165
channel 2 = 1631.335
channel 3 = 957.0699
channel 4 = 360.9018
channel 5 = 242.5404
channel 6 = 76.8999

and did not change to the printed decimal place for a file in Aug 2021 and Feb 2022.

BenjaminTJohnson commented 10 months ago

Ignore this information: I'll probably delete it later when I sort out the correct conversion.
470 nm = 21276.60 cm–1 : 46.8353729 mW/(m^2 sr cm^{-1})-> W/(m^2 sr um) = 996.497495 640 nm = 15625.00 cm–1: 68.4577789 mW/(m^2 sr cm^{-1})-> W/(m^2 sr um) = 1069.652795 860 nm = 11627.91 cm–1 : 73.7946090 mW/(m^2 sr cm^{-1})-> W/(m^2 sr um) = 858.077072 1370 nm = 7299.27 cm–1: 70.4625507 mW/(m^2 sr cm^{-1})-> W/(m^2 sr um) = 514.325182 1600 nm = 6250.00 cm–1: 65.0208639 mW/(m^2 sr cm^{-1})-> W/(m^2 sr um) = 406.380399 2200 nm = 4545.45 cm–1 : 40.0745478 mW/(m^2 sr cm^{-1})-> W/(m^2 sr um) = 182.156853

BenjaminTJohnson commented 10 months ago

Updated ATBD: https://www.star.nesdis.noaa.gov/goesr/documents/ATBDs/Enterprise/ATBD_Enterprise_Cloud_and_Moisture_Imagery_Product_v4_2021-01-13.pdf

Most importantly, page 22, equations 3-2 to 3-5

BenjaminTJohnson commented 10 months ago

@gthompsnJCSDA Using the converter information provided in 3.4.4 of the above document, I now have converted from the SpcCoeff Spectral irradiance values:

2111.779435
1681.038527
988.641475
373.192299
250.863008
79.606007

Which is roughly consistent with what I expected to see.

BenjaminTJohnson commented 9 months ago

[ edited by @gthompsnJCSDA ]

A first version of visible reflectance from CRTM and used within UFO appears to work reasonably well. We identified a couple possible improvements to address in next version(s).

  1. Update the Esun values to match Table 6 of ATBD document, which are also the Esun values found in L1b source files.
  2. Utilize the earth_sun_distance_ratios.txt to improve reflectance produced in CRTM per Eqn. 3-4.
  3. Possibly reduce the number of view angles computed in CRTM (the reason we think there are 7 print statements per channel in debug log above).
  4. Possibly impose an upper limit of cloud optical depth since the real world has multiple scattering.
gthompsnJCSDA commented 9 months ago

Graphics showing the observed 1800UTC 15 Feb 2022 GOES-16 data, followed by H(x), followed by obs minus background (C384), using channel 1, 8km sub-sampled obs data:

g16_8km_channel1_ObsValue

g16_8km_channel1_hofx

g16_8km_channel1_omb

gthompsnJCSDA commented 6 months ago

@BenjaminTJohnson I think it is OK to close this issue now (with the merging of 99). Further issues and PRs can follow for remaining items that require fixing or improving. If you disagree with close it, please just reopen.