search

climate-mirror / datasets

For tracking data mirroring progress
201 stars 18 forks source link

Dataset at ftp:/cddis.gsfc.nasa.gov/slr/products #151

Open nickrsan opened 7 years ago

nickrsan commented 7 years ago

ftp:/cddis.gsfc.nasa.gov/slr/products.

Suggested in a large email containing many urls

clickbg commented 7 years ago

Downloading

clickbg commented 7 years ago

Downloading it right now:

root@zeus:~ # echo ls | lftp 'ftp://cddis.gsfc.nasa.gov/slr/products/' -rw-r--r-- 1 ftp ftp 2767 May 25 2015 00readme drwxr-xr-x 3 ftp ftp 4096 Feb 08 2016 ac drwxrwsr-x 1394 ftp ftp 110592 May 25 2015 ilrs2014_individual_ac_sinex drwxrwxr-x 4 ftp ftp 4096 Dec 03 2002 ilrsac drwxrwsr-x 1394 ftp ftp 110592 May 25 2015 itrf2014 drwxrwsr-x 6 ftp ftp 4096 May 05 2016 orbits drwxrwxr-x 520 ftp ftp 73728 Jan 26 04:30 pos+eop drwxr-xr-x 4 ftp ftp 4096 Sep 29 2015 resource drwx--x--x 15 ftp ftp 4096 Mar 02 2016 test

root@zeus:~ # echo ls | lftp 'ftp://cddis.gsfc.nasa.gov/slr/' -rw-r--r-- 1 ftp ftp 2943 Mar 20 2015 00readme drwxr-sr-x 16 ftp ftp 4096 Dec 29 21:38 cpf_predicts drwxrwsr-x 10 ftp ftp 4096 Jan 15 2013 data drwx-ws--x 9 ftp ftp 4096 Dec 07 2012 lrolr drwxrwsr-x 33 ftp ftp 4096 Jun 14 2016 predicts drwxrwsr-x 10 ftp ftp 4096 Feb 25 2016 products lrwxrwxrwx 1 ftp ftp 19 Jun 23 2010 slreport -> ../reports/slreport lrwxrwxrwx 1 ftp ftp 17 Jul 08 2010 slrlog -> ../reports/slrlog lrwxrwxrwx 1 ftp ftp 18 Jun 23 2010 slrmail -> ../reports/slrmail lrwxrwxrwx 1 ftp ftp 17 Jul 16 2010 slrocc -> ../reports/slrocc lrwxrwxrwx 1 ftp ftp 17 Jul 16 2010 slrscf -> ../reports/slrscf

Ichimonji10 commented 7 years ago

This data set is about 5.5 GB.

$ lftp cddis.gsfc.nasa.gov
lftp cddis.gsfc.nasa.gov:~> cd slr/products           
cd ok, cwd=/slr/products         
lftp cddis.gsfc.nasa.gov:/slr/products> du -s -c -b
5468690771      .                                                                                   
5468690771      total
Ichimonji10 commented 7 years ago

Downloading with wget --mirror --output-file=slr_products.log --no-verbose --limit-rate=1m ftp://cddis.gsfc.nasa.gov/slr/products/.

bkirkbri commented 7 years ago

@Ichimonji10 @clickbg Thanks! Please update when you have complete mirrors. Location of mirror (if public) and hashdeep output too.

Ichimonji10 commented 7 years ago

Downloaded 5.4 GB of data. Download log and hashdeep .csv file available here: http://www.ichimonji10.name/climatemirror/cddis.gsfc.nasa.gov.slr_products/

The actual data isn't available at this time. I'm syncing it between my local nodes for redundancy purposes, and hope to make torrents available this weekend.

donbright commented 7 years ago

attempted mirror: offline, contact me if data is needed hashdeep:offline, contact me if data is needed

redownloaded 2017 2 19 due to hard drive failure

Ichimonji10 commented 7 years ago

@donbright If you want dynamic DNS, I recommend http://freedns.afraid.org/. I've been using them for years for a variety of sites.

blitzdesigner commented 7 years ago

im loading too, gona be here: ftp://ftp.blitzdesigner.info/pub/climatemirror/cddis.gsfc.nasa.gov_slr_products%20Issue_151/

bkirkbri commented 7 years ago

@blitzdesigner Were you able to complete your mirror?

blitzdesigner commented 7 years ago

@bkirkbri yes sorry i finished, im quite busy at work atm :-) mirror is finished, hashdeep-file is there too.