Develop new calibration procedure for Hyperspectral Imagers

[czender]

Description

We discussed pitfalls of current hyperspectral (HS) calibration procedure and agreed to attempt to improve calibration by expanding the temporal/angular resolution of the Spectralon reflectance imagery, utilizing the spectral downwelling irradiance from the Environmental Logger (EL) sensor, and possibly incorporating radiative transfer modeled (RTM) fluxes to extend the effective bandwidth of the EL sensor beyond 1 micron, so the technique can be used for the SWIR imagery.

Details

1. Lemnatec developed the current (someday to be "old") HS calibration procedure, employed since 201609. This procedure has been based on a single set of exposures made 20161021, and does not use any "real time" data from the EL. The script that "reduces" the spectralon exposures to the form used in the HS workflow is here
2. An older HS calibration procedure utilizes "real time" irradiance from the EL. This was never fully implemented due to lack of input data (spectralon imagery in particular, but also spectral irradiance data).
3. The "new" calibration procedure will combine elements from both of the above procedures, and yield reflectance data that better accounts for zenith-angle and cloud effects, which are both large.

Completion Criteria

• [ ] Update documentation for new calibration procedure
• [ ] Update new calibration procedure to account for dark counts
• [ ] #17 Collect diurnal hyperspectral spectralon images for calibration
• [ ] Convert new spectralon calibration images to netCDF
• [ ] Visually inspect then hyperslab images to contain only spectralon panels
• [ ] Compute zenith angle for each image time
• [ ] Create new reduction procedure based on current reduction here
• [ ] Annotate reduced images with zenith angles, store them in HS extractor directory
• [ ] Modify HS calibration to look for spectralon calibration with nearest two zenith angles
• [ ] Interpolate EL irradiance to time of HS image @FlyingWithJerome
• [ ] Interpolate spectralon exposure to zenith angle of HS image
• [ ] Compute RTM fluxes for zenith angle of HS image
• [ ] Extend EL-observed spectral irradiance with RTM irradiance (for SWIR only)
• [ ] Use interpolated irradiance, spectralon to perform new HS calibration
• [ ] Validation and quality control (e.g., derived reflectances < 1.0, match known standards)

[czender]

As y'all can see, I have created this issue as an "epic" for the new calibration procedure. Please comment add modify. In the next few days I'll start to add some sub-issue that correspond to the completion criteria. If you know what needs to be done, please feel free to add those sub-issues yourself and link back to this epic.

[dlebauer]

Issue moved to terraref/computing-pipeline #281 via ZenHub