search

climate-mirror / datasets

For tracking data mirroring progress
201 stars 18 forks source link

PODAAC ERS1 ERS2 #289

Open JeremiahCurtis opened 7 years ago

JeremiahCurtis commented 7 years ago

ftp://podaac-ftp.jpl.nasa.gov/allData/ers1/ ftp://podaac-ftp.jpl.nasa.gov/allData/ers2/

erikfriesen commented 7 years ago

Checking file sizes right now. Will update when complete.

gofrogs2013 commented 7 years ago

The ers1 alone is nearly 30K files and 10K folders. I eventually got kicked out and wasn't able to get a final tally. This would almost certainly be one you would have to do one subfolder at a time.

erikfriesen commented 7 years ago

I ended up having to restart just before i went to be (screwup on my part). It's been runnng about 9 hours and is up to about 125k files in 49k directories.

erikfriesen commented 7 years ago

well, it was still running this afternoon. i cancelled the job and will move on to an easier task for now

gofrogs2013 commented 7 years ago

I'm going one subfolder at a time now. Once I get it done I'll compress the entire thing and upload it.

bkirkbri commented 7 years ago

@gofrogs2013 Good idea. Please update here when complete (even if you only partially mirror it). Thanks!

gofrogs2013 commented 7 years ago

@bkirkbri Sure thing. I've got one folder so far, the first of two folders within 1992, which alone is over 30K files! When done, I'll zip it and post a link.

gofrogs2013 commented 7 years ago

Just finished the long download. I have the files and checksums here: https://1drv.ms/f/s!AmQ9_sDQ2s44jGME-FEiwiRV_vbt