search

openvenues / lieu

Dedupe/batch geocode addresses and venues around the world with libpostal
MIT License
82 stars 23 forks source link

No matches for a pair you'd expect (?) to find matches for #6

Open thisisaaronland opened 6 years ago

thisisaaronland commented 6 years ago

Passing this along, FYI. These two venues were in the same set of candidates to dedupe but lieu didn't seem to pick up on them.

Isamu Noguchi Museum And Garden
32 37 Vernon Blvd Long Island City, Ny 11106
40.76713, -73.93773
Isamu Noguchi Foundation And Garden Museum
3237 Vernon Blvd Long Is City, Ny 11106-4926
40.76713, -73.93773
albarrentine commented 6 years ago

House number doesn't match because of the space. We don't currently touch house numbers except for basic normalization and removing phrases like "#", "House No.", etc.

There are too many variants and edge cases around the world to use a one-size-fits-all approach for spaces/hyphens without losing information. If you know in advance the data set comes from one country or place, removing spaces and hyphens in preprocessing is an option.

thisisaaronland commented 6 years ago

That's good to know. I can update the "prepare" tool accordingly. Are there any other similar rules / gotchas to be aware of or account for?

albarrentine commented 6 years ago

In data sets without lat/lons, postcodes are often useful, so with voter files for instance I've been normalizing to the ZIP5 in the US. Similarly in GB/CA might want to strip spaces as there can be a variety of different formats.

Some of that we may be able to implement on the libpostal side at some point if the postcode format is unique to a country (GB/CA are definitely unique and I don't think anywhere else in the world has DDDDD-DDDD, so might be fine to assume it's the US and match on the first 5).