search

pelias / whosonfirst

Importer for Who's on First gazetteer
MIT License
26 stars 42 forks source link

Got 403 on import resource #479

Closed kmadenski closed 4 years ago

kmadenski commented 4 years ago

Hey team!

I got 403 on https://dist.whosonfirst.org/sqlite/whosonfirst-data-venue-pl-latest.db.bz2 resource. It is intentional?

missinglink commented 4 years ago

Hi @kmadenski, we don't control the whosonfirst.org domain but I've heard from their team that they've been getting a lot of abuse recently which has resulted in increased costs on the project (which is funded through donations).

Could you please check that your scripts are caching correctly and not performing unnecessary requests? I think the 403 responses are there to throttle users and prevent abuse, they are also logging IPs.

I've just started building our own WOF data for Pelias users, it's much more current.

For now it's hosted on my dev server synergy.io but be aware that the domain name may change without warning so I'd avoid hard-coding the domain in any configs for now. https://wof.synergy.io/sqlite/

Feel free to take what you need, please let me know if you find it useful, we are considering adding them as an alternative to the dist.whosonfirst.org at https://geocode.earth/data

kmadenski commented 4 years ago

Thx for great source.

I see that synergy.io convension for meta (folder bundles) is not working with pelias wof importer. It wants whosonfirst-data-ocean-latest.csv. In source we got ocean.csv.

missinglink commented 4 years ago

@kmadenski would you be interested in opening a PR to fix it?

It's a single line change: https://github.com/pelias/wof/blob/master/stream/bundle.js#L68

missinglink commented 4 years ago

closed by https://github.com/pelias/wof/pull/8