search

climate-mirror / datasets

For tracking data mirroring progress
201 stars 18 forks source link

Naval Research Laboratory Marine Meteorology #230

Open nickrsan opened 7 years ago

nickrsan commented 7 years ago

http:/www.usgodae.org/ftp/outgoing.

Suggested in a large email containing many urls

LFenske commented 7 years ago

There is a robots.txt file that prevents wget from mirroring. Does anyone know a good way around it or who to contact to turn it off?

as-com commented 7 years ago

-e robots=off

On Jan 28, 2017, 4:36 PM -0500, LFenske notifications@github.com, wrote:

There is a robots.txt file that prevents wget from mirroring. Does anyone know a good way around it or who to contact to turn it off? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

DiabolicalTMcD commented 7 years ago

Pulling