search

climate-mirror / datasets

For tracking data mirroring progress
201 stars 18 forks source link

GHCN Subset: Daily/all #333

Open gabefair opened 7 years ago

gabefair commented 7 years ago

This dataset is a part of a giant dataset of The Global Historical Climatology Network (GHCN) #331. We needed to break that one up into parts.

Once you are done:

gabefair commented 7 years ago

I have a copy of this dataset FINISHED --2017-03-13 09:03:48-- https://www.dropbox.com/sh/9wthl50iszkmzs2/AAAeLllJU4cezYZ8afOJaf6Da?dl=0

hashes coming soon 25G ./all

HostileGranola commented 7 years ago

I have a copy of this dataset as of 2017-03-21 13:48:00 UTC

Hash results are here

Size results are here

Juerd commented 7 years ago

Slow download, stalls often. 11 GB in 18 hours. Will provide short term public mirror when done.

Juerd commented 7 years ago

http://[2a03:b0c0:2:d0::1dae:1001]/ftp.ncdc.noaa.gov-pub-data-ghcn-daily-all.tar.gz http://188.166.4.6/ftp.ncdc.noaa.gov-pub-data-ghcn-daily-all.tar.gz

Compresses very well, so only offering in compressed form. 26 GB raw, 2.9 GB compressed.

Will stay online until the 1 TB transfer limit is reached (this should allow for 75+ more full mirrors), or a month has passed, whichever comes first.

HostileGranola commented 7 years ago

@Juerd Do you have any hash and file size results for comparison?

x775 commented 7 years ago

I have a complete copy as of this posting.

md5: f8c4ab830a81844cfe99ff65cf5a9749 sha256: 6bca40e58ed61311f5906af5ffe050180488b7e4a797b80080c519a50b860740

Individual md5 checksums: https://gist.github.com/x775/e5da487abd157032952a97835aa2b12e Individual sha256 checksums: https://gist.github.com/x775/c582f82bf9530cac76a7690dec1e7daf

Size: 25.09811GB

Compressed name: GHCN_Subset_Daily_all.7z Compressed md5: 9a5ae8bcf52716c53d45cdceeb0e7998 Compressed sha256: 78711e8ecfef7237a8ab26ba8cf559dffd9c7e1f81f61bca3ca030975f1ece58 Compressed size: 2.1308GB Compressed download link: https://drive.google.com/open?id=0B6PlQrUTwL1Pak1xTG9va1ZJY2c

entr0p1 commented 7 years ago

Downloading this since last night, been going for over 12 hours and only done 4.5GB (100/40 connection). Working on a server to put all this stuff on, will keep you all posted.

x775 commented 7 years ago

It did take a very long time to download for me as well.

entr0p1 commented 7 years ago

@x775 Good to know, thanks! out of interest what are you using to get such precise size measurements? I'm just using "du -sh" at the moment

entr0p1 commented 7 years ago

Got it finally

Checksums: https://gateway.ipfs.io/ipfs/QmUbBhENtU4MAVftvK59FxnMv1L7S2HrvuPJKrXQKXEC7o Root Directory: https://gateway.ipfs.io/ipfs/QmXmgGSszDnBjnLG52QqBx1GByE598geWrqYcYGbJEPXLG Size: 26.05GB