pelias / whosonfirst

Importer for Who's on First gazetteer
MIT License
26 stars 42 forks source link

Allow downloading combined files from any mirror #530

Closed orangejulius closed 2 years ago

orangejulius commented 2 years ago

Background: Over at Geocode Earth we've now set up our own mirror of WOF data that sits a bit closer to our servers. It lets us kick off new builds, etc a bit faster than downloading from the public WOF server we host on Bunny CDN.

When trying that out today, I noticed some logic from ~2 years ago was causing the download script to download not only the combined, planet-level files, but every country file as well.

I think this logic dates back from when we were transitioning off dist.whosonfirst.org, which as I recall didn't have combined files (they had recently moved from one giant Github repo to the per-country repository setup that exists today).

As far as I can tell this logic isn't needed any more, it's fair to assume any WOF mirror would follow the same directory structure as the primary mirror that we host, meaning it would have both combined and country-level files. That's what this PR now allows for.