search

NASA-IMPACT / veda-data

2 stars 0 forks source link

Add recent items to NO2 dataset #115

Closed j08lue closed 1 week ago

j08lue commented 2 months ago

More recent data is available for the NO2 dataset in the VEDA EarthData Dashboard (until end of 2023) and more is coming very soon (Jan-Mar 2024).

There may already be new files in our staging bucket or where else we store the NO2 data.

Acceptance criteria

anayeaye commented 2 months ago

Just for ref, this has been an ongoing need for no2 and co2. These could be good cases for event driven continuous ingests:

ongoing no2 ongoing co2

j08lue commented 3 weeks ago

@ividito I take it we have been working on the event-driven ingest, but we need to fast-track the update of the NO2 dataset, please.

@freitagb said he could run the discovery / ingest, but if you can get to it today, @ividito, please indicate here.

ividito commented 3 weeks ago

no2-monthly and no2-monthly-diff have been updated with new items (up to the end of 2023), but we haven't seen new assets for 2024.

As @j08lue mentioned, we're still finishing work on event-driven ingestion, so future ingests for these collections should happen automatically.

j08lue commented 3 weeks ago

I notified EO Dashboard of the update, btw: https://github.com/eurodatacube/eodash/issues/2590

j08lue commented 3 weeks ago

I can confirm the data is up-to-date through 2023 in STAC and in the VEDA Earthdata Dashboard

j08lue commented 3 weeks ago

Sorry, @ividito, turns out the update also includes the yearly data product, collection OMI_trno2-COG, where a single file was added (for 2023).

https://radiantearth.github.io/stac-browser/#/external/staging-stac.delta-backend.com/collections/OMI_trno2-COG

Is that also already in our dev catalog and can be promoted to production?

Btw, I think it would be great to rename the yearly data collection to something more straightforward, like no2-yearly. I'll create a separate ticket for that.

j08lue commented 1 week ago

Here is the 2023 file to add to the OMI_trno2-COG collection veda-data-store-staging/OMI_trno2-COG/OMI_trno2_0.10x0.10_2023_Col3_V4.tif.

I verified that it is a valid cog and has the same format as the 2022 file.

ividito commented 1 week ago

This data is currently available on the dashboard: https://www.earthdata.nasa.gov/dashboard/data-catalog/no2/explore?projection=mercator%7C%7C&basemapid=satellite&datetime=2023-01-01T00%3A00%3A00.000Z&layer=OMI_trno2-COG&position=26.2948%7C0%7C1.22

j08lue commented 1 week ago

Awesome, thanks!

I notified the EO Dashboard team...