Empty output file when temporal range is too close to the metadata timestamp

[ank1m]

During our L2SS-B-S-C regression tests we noticed StitchEE failures for some of the edge-cases. When we looked deeper, it turned out that the l2ss-py produced file with empty arrays. It happens when the requested temporal subsetting is on the edge of the file timestamp. For example, for granule TEMPO_NO2_L2_V01_20240123T231358Z_S013G03.nc the metadata timestamp is 2024-01-23 23:13:58. So if we request temporal subsetting up to 2024-01-23 23:13:59 (shifted by 1 second) this particular granule is included. But when l2ss-py starts subsetting, the more precise time information in the file has no data points for anything within [, 2024-01-23 23:13:59] time range. The output file has an empty array in it.

• Here is the request that can reproduce this issue: https://harmony.uat.earthdata.nasa.gov/C1254854453-LARC_CLOUD/ogc-api-coverages/1.0.0/collections/%2Fproduct%2Fvertical_column_total/coverage/rangeset?forceAsync=true&subset=time(%222024-01-23T23%3A07%3A00%22%3A%222024-01-23T23%3A13%3A59%22)
• Additionally, the output filename in harmony seems to be different. Notice in the log below that the output file doesn't have _subsetted suffix in the filename:

  {
  "status": "successful",
  "message": "The job has completed successfully",
  "progress": 100,
  "createdAt": "2024-03-29T16:50:07.260Z",
  "updatedAt": "2024-03-29T16:50:21.692Z",
  "dataExpiration": "2024-04-28T16:50:07.260Z",
  "links": [
  {
  "title": "STAC catalog",
  "href": "https://harmony.uat.earthdata.nasa.gov/stac/bf48bf3c-894a-466a-b02d-b84fd1385c85/",
  "rel": "stac-catalog-json",
  "type": "application/json"
  },
  {
  "href": "https://harmony.uat.earthdata.nasa.gov/service-results/harmony-uat-staging/public/bf48bf3c-894a-466a-b02d-b84fd1385c85/4055633/TEMPO_NO2_L2_V01_20240123T230041Z_S013G01_product_vertical_column_total_subsetted.nc4",
  "title": "TEMPO_NO2_L2_V01_20240123T230041Z_S013G01_product_vertical_column_total_subsetted.nc4",
  "type": "application/x-netcdf4",
  "rel": "data",
  "bbox": [
  -89.4,
  17.2,
  -87,
  57.7
  ],
  "temporal": {
  "start": "2024-01-23T23:00:41.000Z",
  "end": "2024-01-23T23:07:21.000Z"
  }
  },
  {
  "href": "https://harmony.uat.earthdata.nasa.gov/service-results/harmony-uat-staging/public/bf48bf3c-894a-466a-b02d-b84fd1385c85/4055634/TEMPO_NO2_L2_V01_20240123T230721Z_S013G02_product_vertical_column_total_subsetted.nc4",
  "title": "TEMPO_NO2_L2_V01_20240123T230721Z_S013G02_product_vertical_column_total_subsetted.nc4",
  "type": "application/x-netcdf4",
  "rel": "data",
  "bbox": [
  -98.7,
  17.2,
  -88.1,
  57.7
  ],
  "temporal": {
  "start": "2024-01-23T23:07:21.000Z",
  "end": "2024-01-23T23:13:58.000Z"
  }
  },
  {
  "href": "https://harmony.uat.earthdata.nasa.gov/service-results/harmony-uat-staging/public/bf48bf3c-894a-466a-b02d-b84fd1385c85/4055635/TEMPO_NO2_L2_V01_20240123T231358Z_S013G03_product_vertical_column_total.nc4",
  "title": "TEMPO_NO2_L2_V01_20240123T231358Z_S013G03_product_vertical_column_total.nc4",
  "type": "application/x-netcdf4",
  "rel": "data",
  "bbox": [
  -111.0561523,
  17.2004852,
  -95.2178192,
  58.26931811
  ],
  "temporal": {
  "start": "2024-01-23T23:13:58.000Z",
  "end": "2024-01-23T23:20:35.000Z"
  }
  },
  {
  "href": "https://harmony.uat.earthdata.nasa.gov/jobs/bf48bf3c-894a-466a-b02d-b84fd1385c85?page=1&limit=2000",
  "title": "The current page",
  "type": "application/json",
  "rel": "self"
  }
  ],
  "request": "https://harmony.uat.earthdata.nasa.gov/C1254854453-LARC_CLOUD/ogc-api-coverages/1.0.0/collections/%2Fproduct%2Fvertical_column_total/coverage/rangeset?forceAsync=true&subset=time(%222024-01-23T23%3A07%3A00%22%3A%222024-01-23T23%3A13%3A59%22)",
  "numInputGranules": 3,
  "jobID": "bf48bf3c-894a-466a-b02d-b84fd1385c85"
  }

[ank1m]

This issue has been investigated before, and no clear consensus was reached. The discussion with harmony community is still on-going and things might change in the future.