search

SlideRuleEarth / sliderule

Server and client framework for on-demand science data processing in the cloud
https://slideruleearth.io
Other
25 stars 11 forks source link

Invalid header signature error returned when reading some ATL03 granules #132

Closed jpswinski closed 5 months ago

jpswinski commented 1 year ago

During @slhowardESR's runs, some reads of granules fail with the error that the header signature in the file is incorrect.

2022-08-24 12:40:35 | ip=35.90.219.155 level=critical caller=H5Coro.cpp:3566 msg="Failure reading nsidc-s3://ATL03_20191219021259_12700510_005_01.h5//gt3l/geolocation/segment_ph_cnt: invalid header signature: 0x45455254 (/gt3l/geolocation/segment_ph_cnt)"
2022-08-24 12:40:35 | ip=35.90.219.155 level=critical caller=H5Coro.cpp:3566 msg="Failure reading nsidc-s3://ATL03_20191219021259_12700510_005_01.h5//gt3l/geolocation/reference_photon_lon: invalid header signature: 0x45455254 (/gt3l/geolocation/reference_photon_lon)"
2022-08-24 12:40:35 | ip=35.90.219.155 level=critical caller=H5Coro.cpp:3566 msg="Failure reading nsidc-s3://ATL03_20191219021259_12700510_005_01.h5//gt3l/geolocation/reference_photon_lat: invalid header signature: 0x45455254 (/gt3l/geolocation/reference_photon_lat)"
2022-08-24 12:40:35 | ip=35.90.219.155 level=critical caller=H5Coro.cpp:3566 msg="Failure reading nsidc-s3://ATL03_20191219021259_12700510_005_01.h5//gt2l/geolocation/segment_ph_cnt: invalid header signature: 0x45455254 (/gt2l/geolocation/segment_ph_cnt)"
2022-08-24 12:40:35 | ip=35.90.219.155 level=critical caller=H5Coro.cpp:3566 msg="Failure reading nsidc-s3://ATL03_20191219021259_12700510_005_01.h5//gt1l/geolocation/segment_ph_cnt: invalid header signature: 0x100008 (/gt1l/geolocation/segment_ph_cnt)"
2022-08-24 12:40:35 | ip=35.90.219.155 level=critical caller=H5Coro.cpp:3566 msg="Failure reading nsidc-s3://ATL03_20191219021259_12700510_005_01.h5//gt1l/geolocation/reference_photon_lat: invalid header signature: 0x100008 (/gt1l/geolocation/reference_photon_lat)"
2022-08-24 12:40:35 | ip=35.90.219.155 level=critical caller=H5Coro.cpp:3566 msg="Failure reading nsidc-s3://ATL03_20191219021259_12700510_005_01.h5//gt1l/geolocation/reference_photon_lon: invalid header signature: 0x100008 (/gt1l/geolocation/reference_photon_lon)"
2022-08-24 12:40:35 | ip=35.90.219.155 level=critical caller=H5Coro.cpp:3566 msg="Failure reading nsidc-s3://ATL03_20191219021259_12700510_005_01.h5//gt2l/geolocation/reference_photon_lon: invalid header signature: 0x45455254 (/gt2l/geolocation/reference_photon_lon)"
2022-08-24 12:40:35 | ip=35.90.219.155 level=critical caller=H5Coro.cpp:3566 msg="Failure reading nsidc-s3://ATL03_20191219021259_12700510_005_01.h5//gt2l/geolocation/reference_photon_lat: invalid header signature: 0x45455254 (/gt2l/geolocation/reference_photon_lat)"
jpswinski commented 1 year ago

Screenshot from 2022-11-17 14-41-41

jpswinski commented 1 year ago

I appears that the files in question are missing the track subgroups (gt1l, gt1r, gt2l, gt2r, gt3l, gt3r).

jpswinski commented 1 year ago

SlideRule can do a sanity check on the integrity of the atl03 granule before moving ahead with reading it, so that a better error message can be provided to the user; but that will slow down the processing of each request.

I recommend that we document this, and leave as is.

jpswinski commented 1 year ago

It may be related to TEP stares... @tsutterley discovered that they correlate to when the spacecraft is doing a TEP stare

jpswinski commented 5 months ago

Jeff Lee said that if no data is provided for a beam, then the corresponding group will not be present. So for all-ocean granules, the weak beams will be missing and this is the error we see.

This can be closed as the TEP stare and the ocean granules explain this behavior.