search

pangeo-cmip6 / sync

Workflows to keep CMIP6 data synchronized between GCS and S3 storage
2 stars 1 forks source link

Generate catalogs directly from S3 contents #5

Open naomi-henderson opened 3 years ago

naomi-henderson commented 3 years ago

I am now writing scripts to directly create the CMIP6 catalogs from the contents of s3://cmip6-pds/CMIP6 . This will assure us that the catalog matches the actual holdings. In addition, we can have a better way to check them against each other in the future

Once this is working, we will need to adjust the script in the catalog job of the GH action script

rabernat commented 3 years ago

I so deeply appreciate your work on this Naomi!

And I hope you know that this is not nearly urgent enough to justify working on a Saturday! 😉 Unless you're enough of a nerd (like me 🤓 ) to enjoy coding in your free time...