leap-stc / data-management

Collection of code to manually populate the persistent cloud bucket with data
https://catalog.leap.columbia.edu/
Apache License 2.0
0 stars 6 forks source link

New Dataset [eNATL60-TSW-600m] #147

Open auraoupa opened 2 months ago

auraoupa commented 2 months ago

Dataset Name

eNATL60 TSW 600m

Dataset URL

https://ige-meom-opendap.univ-grenoble-alpes.fr/thredds/catalog/meomopendap/extract/MEOM/eNATL60/eNATL60-BLBT02/1d/eNATL60/catalog.html

Description

eNATL60-TSW-600m is an extraction of a very high resolution oceanic simulation of the North Atlantic performed at MEOM, IGE (FRANCE) that will help design sub-grid parametrization based on machine learning in M2LINES framework

Size

The dataset consists in 365 files of 410M, so a total of 147G

License

Creative Commons Zero v1.0 Universal

Data Format

NetCDF

Data Format (other)

No response

Access protocol

HTTP(S)

Source File Organization

365 daily files (one record per file) from 2009, July the 1st to 2010, June the 30th

Example URLs

https://ige-meom-opendap.univ-grenoble-alpes.fr/thredds/fileServer/meomopendap/extract/MEOM/eNATL60/eNATL60-BLBT02/1d/eNATL60/eNATL60-BLBT02_y2010m06d30.1d_TSWm_600m.nc

Authorization

None

Transformation / Processing

Concatenation along the time dimension

Target Format

Zarr

Comments

No response

norlandrhagen commented 2 months ago

Hey there @auraoupa,

This seems like a great candidate. A few follow-up questions.

Thanks!

auraoupa commented 2 months ago

Hi @norlandrhagen,

The answers are :

A note : The last time (for the 60m dataset) we used zenodo to host the data, if anything is going wrong due to the opendap, I can host this data too on zenodo.

Thanks for the quick follow-up !

norlandrhagen commented 2 months ago

Thanks for the reply!

Good to know about the TSWm and thanks for removing the others. I think opendap should work, but I'll let you know if we run into issues and need them hosted on Zenodo.

Glad the 60m dataset is looking good! If you see anything you want to change about it in the catalog (description, naming etc.) it's super simple, so just let us know.