pangeo-data / pangeo-datastore

Pangeo Cloud Datastore
https://catalog.pangeo.io
48 stars 16 forks source link

Add regional llc4320 cut-outs to Pangeo #127

Open kdrushka opened 3 years ago

kdrushka commented 3 years ago

Hello,

As part of the SWOT Science Team and related Adopt-a-Crossover effort, I am leading a project to develop code and facilitate analysis of the the MITgcm llc4320 simulation in preparation for a bunch of regional field campaigns planned to take place after the SWOT launch. llc4320 data from 10 regions have been extracted thanks to @menemenlis, and converted to netCDF and made available on the AWS cloud via PO.DAAC as the “Pre-SWOT Ocean Simulation LLC4320” thanks to @jinbow.

These data will be used by members of the SWOT community for planning field campaigns, comparing to regional/global simulations, and hopefully many other cool analysis projects. The beauty of the regional cut-outs is that they are MUCH more manageable than the global llc4320 due to their relatively small size (regions are 4 x 4 degrees). Plus, since the files have been converted from the binary/LLC grid format, and all variables (including 3D fields) are in a single file, and a NAS account isn't needed, the bar to access these data has been greatly lowered. Hurrah! Having these files available on AWS / PO.DAAC is a huge step toward making them accessible to the international community, but to facilitate even broader use and code-sharing it would be fantastic if the files were accessible through Pangeo.

Each of the 10 regions has 429 daily data files that include five full-depth 3D variables & nine 2D variables at a 1-hour time step. Individual files are are ~0.5 to 2 GB each, for a total of ~5 TB for all files in all 10 regions. Detailed documentation is here.

Any thoughts on whether adding these data to the Pangeo catalog is possible? If so, please advise on what the next steps might be. I’m super new to this but eager to make it work.

Many thanks, Kyla