ajnisbet / opentopodata

Open alternative to the Google Elevation API!
https://www.opentopodata.org
MIT License
327 stars 71 forks source link

Suggestion of code to bulk download DEM data more easily from USGS #71

Closed SamDurand closed 1 year ago

SamDurand commented 1 year ago

I had trouble downloading the USGS data, even with the correct login information and following earthdata's advices.

But once I was authenticated correctly to the earthdata site with my account and was able to download any file manually, this code worked well for me.

Just create a text file from srtm30m_urls.txt (for example) and execute the code below with your paths to the Chrome app and data_urls.txt.

import webbrowser
import time

# Windows
chrome_path = "C:/Program Files/Google/Chrome/Application/chrome.exe %s"

# MacOS
# chrome_path = 'open -a /Applications/Google\ Chrome.app %s'
# Linux
# chrome_path = '/usr/bin/google-chrome %s'

with open("data_urls.txt", "r") as f:
    url_list = f.read().split("\n")

for i in range(len(url_list)):
    webbrowser.get(chrome_path).open_new_tab(url_list[i])
    if i % 100 == 0:
        time.sleep(5) #pause 5s every 100 it.
ajnisbet commented 1 year ago

Thanks for the suggestion, this is a great way to deal with tricky authentication!

I simplified the script to:

import webbrowser
import time

with open("srtm30m_urls.txt", "r") as f:
    url_list = f.read().split("\n")

for i, url in enumerate(url_list):
    webbrowser.open_new_tab(url)
    if i % 100 == 0:
        time.sleep(5) # pause 5s every 100 it to avoid rate limiting.

which I think will do much the same thing. And if you're ok with me mentioning you by [your github] name and linking to your github profile let me know and I'll add that attribution to the docs too!

SamDurand commented 1 year ago

Of course! Thanks for your great work!