search

climate-mirror / datasets

For tracking data mirroring progress
201 stars 18 forks source link

Coupled Model Intercomparison Project #85

Open nickrsan opened 7 years ago

nickrsan commented 7 years ago

Name: Coupled Model Intercomparison Project Organization: Description URL: http://cmip-pcmdi.llnl.gov/cmip5/data_portal.html Download URL: File Types: Size: Status:

johncronan commented 7 years ago

HTTP hosting for CMIP5 is all at http://esgf1.dkrz.de/thredds/catalog/catalog.html

That's the German Climate Computing Center. Probably safe.

See attached doc: hopefully it's enough to make sense of the filenames, so that the LLNL index is not necessary. cmip5_data_reference_syntax.pdf

ktoddbrown commented 7 years ago

While most of the data is mirrored on other repo's (not just the .de link provided) not all data is on all mirrors. The llnl.gov site is the main gateway.

This is also a LOT of data, CMIP5 has several PB of data and that's not including the previous CMIPs that are also out there.

Still I would suggest this is low priority. Modeling centers and PIs are generally happy to provide these runs when asked directly. Plus they could, in theory, be reconstructed from driving files and code base. Though in practice this would be extremely difficult in many cases due to the idiosyncratic nature of the code base.

donbright commented 7 years ago

Size estimates (unsure of exact path, chose pub/dcp/archive):

lftp gdo-dcp.ucllnl.org:/pub/dcp/archive> du -h -d 1
2.2T    ./bcca                                                                 
312G    ./bcsd                                                                 
31T     ./cmip5                                                                
1.3T    ./OBS                                                                  
5.1G    ./obs                                                                  
1.6T    ./wwcra                                                                
36T .