NASA-IMPACT / veda-data-pipelines

data transformation - ingestion - publication pipelines to support VEDA
Other
13 stars 6 forks source link

Add OMPS Ozone data layer to a dashboard (high-level steps) #65

Closed abarciauskas-bgse closed 2 years ago

abarciauskas-bgse commented 2 years ago
    • [x] Identify dataset and where it will be accessed from. Check it's a good source with science team. Ask about specific variables and required spatial and temporal extent. Note most datasets will require back processing (e.g. generating cloud-optimized data for historical data).

      Details about the OMPS dataset were provided over email by David Haffner:

      OMPS total ozone for the dashboard: Since the plan is to use the daily L3 gridded files and we make those files publicly available, I think it would streamline the process if you could pull the latest data directly from the GES DISC routinely. The daily files are available here: https://snpp-omps.gesdisc.eosdis.nasa.gov/data/SNPP_OMPS_Level3/OMPS_NPP_NMTO3_L3_DAILY.2 Information about the dataset is at the DISC site too: https://disc.gsfc.nasa.gov/datasets/OMPS_NPP_NMTO3_L3_DAILY_2/summary?keywords=NMTO3

      You want to get the data from “ColumnAmountO3” field in the files. The data are stored in HDF5 format so they’re easy to read, and the metadata in the file will give you the coordinate orientation.

    • [ ] If the dataset is ongoing (i.e. new files are continuously added and should be included in the dashboard), design and construct the forward-processing workflow.
      • Each collection will have a workflow which includes discovering data files from the source, generating the cloud-optimized versions of the data and writing STAC metadata.
      • Each collection will have different requirements for both the generation and scheduling of these steps, so a design step much be included for each new collection / data layer.
    • [ ] Verify the COG output with the science team by sharing in a visual interface.
    • [ ] Verify the metadata output with STAC API developers and any systems which may be depending on this STAC metadata (e.g. the front-end dev team).
    • [ ] If the dataset should be backfilled, create and monitor the backward-processing workflow.
    • [ ] Engage the science team to add any required background information on the methodology used to derive the dataset.
    • [ ] Add the dataset to the production dashboard.

Additional information about the color scale from David H:

For mapping the data, I recommend using this color scale, image

This is the standard color scale from NASA’s Ozone Watch website. We try to use it consistently. A nice example showing it in use with the OMPS data is here, at the ozonewatch site, showing Southern Hemisphere total ozone in September of last year when the ozone hole was going strong, https://ozonewatch.gsfc.nasa.gov/monthly/monthly_2021-09_SH.html

Values are higher in the Northern Hemisphere in the springtime when the meteorology is very dynamic, https://ozonewatch.gsfc.nasa.gov/monthly/monthly_2021-03_NH.html

In addition to polar projections, we also do global maps like this, https://ozonewatch.gsfc.nasa.gov/ozone_maps/images/Y2022/M01/OZONE_D2022-01-24_G%5e716X363.IOMPS_PNPP_V21_MGEOS5FP_LGL.PNG

abarciauskas-bgse commented 2 years ago

Going to close this for now since we aren't planning on including this dataset in V0