search

climate-mirror / datasets

For tracking data mirroring progress
201 stars 18 forks source link

NOAA NCEP Modeled Wave Height Hindcasts Data #121

Open jebyrnes opened 7 years ago

jebyrnes commented 7 years ago

Dataset Homepage: ftp://polar.ncep.noaa.gov/pub/history/waves/ Dataset Bulk Download Link (if available): None Identified Mirror URLs: Mirrored By: Size: ~ 525 GB Responsible Agency: NOAA See Also: http://polar.ncep.noaa.gov/waves/download.shtml, http://polar.ncep.noaa.gov/waves/product_table.shtml?-multi_1-

zenoreilly commented 7 years ago

Downloading this one now. Can someone explain how to mark as in-progress etc pls?

nbyloff commented 7 years ago

Pulling everything from the FTP currently. I am also using HTTrack to download the site from the other two links. Is that sufficient?

jebyrnes commented 7 years ago

That’s fantastic. Thank you. Trying to get enough server space of my own to do some of this work, but so far cannot.

On Jan 24, 2017, at 4:57 PM, Nathan Byloff notifications@github.com<mailto:notifications@github.com> wrote:

Pulling everything from the FTP currently. I am also using HTTrack to download the site from the other two links. Is that sufficient?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/climate-mirror/datasets/issues/121#issuecomment-274952077, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AAuOol3mLJHCqJqs4f3qpwVGyXOLYXcUks5rVnPBgaJpZM4LqPxP.

nbyloff commented 7 years ago

Good deal. I am not sure I will crawl the whole site now that I look at it. It seems like I just need to grab all the data from the FTP sites? For example, I will mirror these:

ftp://polar.ncep.noaa.gov/pub/history/waves/ ftp://ftpprd.ncep.noaa.gov/pub/data/nccf/com/wave/prod/ ftp://polar.ncep.noaa.gov/pub/waves/develop ftp://polar.ncep.noaa.gov/pub/waves/ ftp://polar.ncep.noaa.gov/pub/waves/egowafs_fd

One of the links on the Model Data Access tab points to this site: http://nomads.ncep.noaa.gov/. This site probably needs it's own issue assigned (if there isn't one already). Please note, This is the one link on the Model Data Access tab I am not pulling from. There is a ton of data on this site.

zenoreilly commented 7 years ago

I have a large chunk of this data (441 GB) downloaded already at http://destroyernet.com/polar.ncep.noaa.gov . I mentioned 3 days ago that I was starting it; why didn't anyone answer? Also, the Google doc is gone.

bkirkbri commented 7 years ago

@zeno0771 Thank you for mirroring this. We aim for multiple mirrors per issue, so it is not a problem that others are downloading. I'll mark as in progress.

ghost commented 7 years ago

Concur. We, at The Azimuth Backup Project, have nothing for this.