Get list of EMIT validation products from EMIT team

[ngachung]

Need to get a set of or access to the validaiton products that the EMIT team will be using. These will be required for U-DS to populate the data catalog and store in an EMIT venue bucket.

Blocks https://github.com/unity-sds/unity-data-services/issues/354

[mike-gangl]

From Godwins notes:

Figure out how to download EMIT netcdf products from LP-DAAC to Jupyter - bug Mike and Nga

Phil suggested using CMR, which is how we do it as well so this is perfect CMR query could be done using a REGEX and EMIT time-stamp (this should be unique enough for CMR query to work). FID in their HPC only has time-stamp Jay would like to identify 100-200 cases and identify by FID FID (flightline ID) is the scene unique identifier, Jay would only know the time-stamp Jay will need to give us the FIDs that he wants Want L1B RAD and OBS files Jay can re-construct the orbit number Have Jay learn to work with CMR and such using our Unity notebook

[mike-gangl]

Still in progress as of 4/24/2024

Ah, so these are the EMIT L1B files you start from?

We used these same files on our SISTER work- for example one set of inputs comes from [https://cmr.earthdata.nasa.gov/search/granules.stac?collection_concept_id=C2408009906-LPCLOUD&temporal[]=2023-08-10T03:41:03.000Z,2023-08-10T03:41:03.000Z](https://cmr.earthdata.nasa.gov/search/granules.stac?collection_concept_id=C2408009906-LPCLOUD&temporal%5b%5d=2023-08-10T03:41:03.000Z,2023-08-10T03:41:03.000Z)

Which gives us:
EMIT_L1B_RAD_001_20230810T034101_2322203_002.nc
EMIT_L1B_OBS_001_20230810T034101_2322203_002.nc

We can either pull these and have them ready locally, or we can pull them at run-time. IF you want to run against all of them, we’d probably create a wrapper to make these calls/executions anyway.

-Mike

Response:

Yes, that’s right. There’s several that are needed, not just the two you list here.

I’m asking around about a list of cases. If no one has a preferred list, I’m just going to select 200 plume cases.

[mike-gangl]

@ngachung - The list of files needed for the plume detection is attached.

For each entry, there is a column (line_name) that looks like this: emit20230620t084426.

To find the corresponding file in CMR, we search on the collection (C2408009906-LPCLOUD) for that pattern in the granule_readbale_name:

https://cmr.earthdata.nasa.gov/search/granules.json?echo_collection_id=C2408009906-LPCLOUD&options[readable_granule_name][pattern]=true&page_num=1&page_size=20&readable_granule_name[]=*20230620T084426*&sort_key=-start_date

20240221_plume_list_CM.xlsx

[mike-gangl]

Closing as we have the list of items. We should probably bring them into the system and then ensure things like bbox and searches are working.

We might also want to see if there is "custom" metadata that would be helpful (e.g. the plume lat/lon locations?)