search

pangeo-data / pangeo-datastore

Pangeo Cloud Datastore
https://catalog.pangeo.io
48 stars 16 forks source link

Add CCMP winds to Pangeo #112

Closed cgentemann closed 4 years ago

cgentemann commented 4 years ago

Hi. @willirath and I both seem to use CCMP winds & have both put into Zarr.
I think they would be helpful to have on Pangeo.
The dataset goes from 7/1987 - 7/2020 and is 273 GB. It is 25km x 25 km x 6 hourly data. Zarr store chunks are 2000(time)x157(lat)x180(lon) which are ~150MB files. A nice description of the dataset is here: https://climatedataguide.ucar.edu/climate-data/ccmp-cross-calibrated-multi-platform-wind-vector-analysis?qt-climatedatasetmaintabs=1#qt-climatedatasetmaintabs

rabernat commented 4 years ago

👍 Great! We are happy to support this.

Eventually, pangeo-forge will be able to automate this ingestion. For now, it's still a manual process.

@charlesbluca - can you please set up a new bucket for this and work with @cgentemann to get the data uploaded and catalogged?

charlesbluca commented 4 years ago

Sure! @cgentemann, how does gs://pangeo-ncar-ccmp sound for a bucket name? From there, I can give you any additional privileges needed to upload the data there.

cgentemann commented 4 years ago

CCMP sponsored by MEASURES NASA program, how about gs://pangeo-nasa-ccmp ?

charlesbluca commented 4 years ago

That works! I just created the bucket and can give you permissions - do you have an account associated with Pangeo's Google Cloud project?

cgentemann commented 4 years ago

great, thanks! moving conversation to gitter.

cgentemann commented 4 years ago

CCMP added to pangeo data store & master intake cat updated. tutorial: https://github.com/cgentemann/cloud_science/blob/master/tutorials/Access_CCMP_pangeo.ipynb