AliFlux / MapTilesDownloader

A super easy to use map tiles downloader built using Python
MIT License
427 stars 125 forks source link

HTTP Response 429 (Too Many Requests) if I select Open Street Maps #7

Open DiHLoS opened 4 years ago

DiHLoS commented 4 years ago
...
127.0.0.1 - - [29/Apr/2020 00:15:04] "POST /download-tile HTTP/1.1" 200 -
HIT: https://a.tile.openstreetmap.org/{z}/{x}/{y}.png
RETURN: 429
...

You need find a way for add header at request 'urllib.request.urlretrieve', like this: {"User-Agent", "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:47.0) Gecko/20100101 Firefox/47.0"} ? Your code should seem like a Webbrowser for OSM servers, not a script.

AliFlux commented 4 years ago

I'm not sure if it goes against OSM terms and conditions

DiHLoS commented 4 years ago

I made a small example. I try to download the OSM tile in two ways:

Only the second method works with OSM. See code and result below.

Code:

import requests

url = "https://a.tile.openstreetmap.org/1/0/0.png"

print("Try download OSM tile without 'headers'...")
r = requests.get(url)
print(r)
if r.status_code == 200:
    with open("0.png", "wb") as f:  # saves '0.png'
        f.write(r.content)

print("Try download OSM tile with 'headers'...")
r = requests.get(url, headers={"User-Agent": "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:47.0) Gecko/20100101 Firefox/47.0"})
print(r)
if r.status_code == 200:
    with open("0_.png", "wb") as f:  # saves '0_.png'
        f.write(r.content)

Result:

Try download OSM tile without 'headers'...
<Response [429]>
Try download OSM tile with 'headers'...
<Response [200]>

If you change url to another tile server, for example: https://maps.wikimedia.org/osm-intl/1/0/0.png, both download methods are works properly:

Try download OSM tile without 'headers'...
<Response [200]>
Try download OSM tile with 'headers'...
<Response [200]>

OSM server does not allow to download the tile if no headers is specified...

JaffaKetchup commented 2 years ago

Note that it is against OSM tiles server rules to bulk download tiles like this.