Open asteiker opened 8 months ago
@andypbarrett to provide key variables, including quality flags, needed for higher level products. Utilize ATBD and earthdata-varinfo to inform what information is needed for a controlled ATL03 HDF5 test file.
I've looked at ATL07 and ATL06. Below are a list of candidate variables. I've left out geophysical corrections that are already included in _hph. Paths are from ATL03 data dictionary. Variable lists are from table 2 in ATL07 ATBD and table 5-1 in ATL06 ATBD.
11 out of the 13 variables in /gtx/heights are used by one or both of ATL07 and ATL06, and 12 out of 37 variables from /gtx/geolocation are used.
ATL03 Variable | Description | Data type | Data Rate | Source |
---|---|---|---|---|
_/gtx/heights/deltatime | Elapsed GPS seconds | Double(:) | Photon Detection Rate | 6 + 7 |
_/gtx/heights/dist_phacross | Distance off RGT | Float(:) | Photon Detection Rate | 6 + 7 |
_/gtx/heights/dist_phalong | Distance from equator crossing | Float(:) | Photon Detection Rate | 6 + 7 |
_/gtx/heights/hph | Photon WGS84 height | Float(:) | Photon Detection Rate | 6 + 7 |
_/gtx/heights/latph | Latitude | Double(:) | Photon Detection Rate | 6 + 7 |
_/gtx/heights/lonph | Longitude | Double(:) | Photon Detection Rate | 6 + 7 |
_/gtx/heights/pce_mframecnt | PCE major frame counter | UINT(:) | Photon Detection Rate | 6 |
_/gtx/heights/ph_idchannel | Receive channel id | UINT(:) | Photon Detection Rate | 6 |
_/gtx/heights/ph_idpulse | laser pulse counter id | UINT(:) | Photon Detection Rate | 6 |
_/gtx/heights/qualityph | Photon quality | INT | Photon Detection Rate | 7 |
_/gtx/heights/signal_confph | Photon signal confidence | INT | Photon Detection Rate | 6 |
_/gtx/geolocation/deltatime | Elapsed GPS seconds | DOUBLE(:) | Segment rate | 6 |
_/gtx/geolocation/ph_indexbeg | Index of first segment photon | INT | Photon Detection Rate | 6 |
_/gtx/geolocation/podppdflag | Geolocation quality | INT | Photon Detection Rate | 6 |
_/gtx/geolocation/segment_distx | Segment distance from equator | Double(:) | Segment rate | 6 |
_/gtx/geolocation/segmentid | along-track segment id | INT(:) | Segment rate | 6 + 7 |
_/gtx/geolocation/segmentlength | along-track segment length | Double(:) | Segment rate | 6 + 7 |
_/gtx/geolocation/segment_phcnt | Number of photons | INT(:) | Segment rate | 6 |
_/gtx/geolocation/sigmaacross | across-track geolocation accuracy | Float(:) | Segment rate | 6 + 7 |
_/gtx/geolocation/sigmaalong | along-track geolocation accuracy | Float(:) | Segment rate | 6 + 7 |
_/gtx/geolocation/sigmah | height uncertainty | (Float(:) | Segment rate | 6 |
_/gtx/geolocation/surftype | Surface type | INT(:, 5) | Segment rate | 7 |
_/gtx/geolocation/velocitysc | spacecraft xyz velocity | Float(:,3) | Segment rate | 6 |
We determined that we will grab all variables within the /heights groups for each beam group. A new task was added to actually perform the subsetting using icepyx
.
This may still be helpful to determine core variables needed to create a derived product if we were to reformat to something like geoparquet.
To better control these performance tests, we ought to create stripped down ATL03 files based on information from #23 and #26. The goal is to better control for the complexities in the native data. We can then reduce the noise as we investigate switches in the HDF5 library to increase performance in the cloud.