search

leap-stc / data-management

Collection of code to manually populate the persistent cloud bucket with data
https://catalog.leap.columbia.edu/
Apache License 2.0
0 stars 6 forks source link

Catalog Feature: Date last updated for every store #100

Open jbusecke opened 7 months ago

jbusecke commented 7 months ago

Not urgent, just collecting this here. It would be nice to display the date a store was last updated. I have my template set up to write the git hash and time of recipe execution into the store itself. I wonder if this could be parsed during the catalog creation. cc @andersy005

norlandrhagen commented 7 months ago

I like this organizational structure @jbusecke

andersy005 commented 3 weeks ago

@jbusecke, this is definitely something worth adding to the catalog. i'm curious... can you point me to a dataset/store with this information? i would like to tinker with it and see how we might integrate it with the catalog

jbusecke commented 2 weeks ago

I think the eNATL stores should have this baked in already: https://catalog.leap.columbia.edu/feedstock/enatl

andersy005 commented 2 weeks ago

I think the eNATL stores should have this baked in already: catalog.leap.columbia.edu/feedstock/enatl

unfortunately, the stores are private

jbusecke commented 2 weeks ago

Oh sorry I thought we would have this already on the pod. https://github.com/leap-stc/chirps_feedstock will definitely be on the pod. I am rerunning that now!

jbusecke commented 2 weeks ago

This store is public and has a date stamped (it will soon be replaced by the full time version:

xr.open_dataset("https://nyu1.osn.mghpcc.org/leap-pangeo-pipeline/chirps_feedstock/chirps-global-daily.zarr", engine='zarr')