search

simonw / download-tiles

Download map tiles and store them in an MBTiles database
https://datasette.io/tools/download-tiles
Apache License 2.0
28 stars 6 forks source link

Add new CLI option for custom request headers #7

Open VaclavTrpisovsky opened 1 year ago

VaclavTrpisovsky commented 1 year ago

When downloading from mapy.cz, any request will fail with 403: Forbidden.
My command (downloads hiking maps for most of the Czech Republic): download-tiles cr.mbtiles --zoom-levels=6-15 --bbox="12.1398,48.6183,17.5231,50.9722" --attribution="" --tiles-url=https://mapserver.mapy.cz/turist-m/{z}-{x}-{y}

Turns out, Mapy.cz implemented protection against bulk downloads by only allowing requests with a specific referer. This can be circumvented by changing line 153 in cli.py from
tiles_headers={"User-Agent": user_agent}, to
tiles_headers={"Referer": "https://mapy.cz/"},

Could somebody implement a way to do this without modifying the source code? An option such as --header={"Name:" "Value"} would be appreciated. This is important because devs of their Android app keep killing support for older devices and their offline maps, requiring us to resort to measures like these. Anyway, I hope anyone seeking good offline hiking maps finds this useful. Cheers, VT

VaclavTrpisovsky commented 5 months ago

I made a fork of this repo that includes a --referer option but it was a quick-and-dirty fix, please don't pull it. It seems to work, though. A guide for downloading from Mapy.cz is included there in the README.