opendata / Open-Data-Needs

An ongoing effort to catalog the holes in the open data ecosystem. [RETIRED]
15 stars 0 forks source link

Geocoding of addresses #2

Open waldoj opened 10 years ago

waldoj commented 10 years ago

I'm not aware of any open, unencumbered, accurate tool for geocoding addresses. (e.g., Google's geocoder may only be used to plot points on a Google Map, as per their terms of service.) Here's a great list of available tools, with a bit of information about each one. The best option is an API-based service run by TAMU, which is much better than any other tools, but it's API-based, rather than being an open stack or a downloadable database.

spara commented 10 years ago

Include Data Science Tool Kit, you can use the online version or download a VM if you want to run it locally. Runs via API (also drop-in replacement for Google Style Geocoder) or batch. Also, there are a number of other interesting geo tools included.

Here's an example of using the the DSTK geocoder in an app

paulfitz commented 10 years ago

Missing on the TAMU list is @petewarden's datasciencetoolkit, it has a handy drop-in google-style geocoder. Can't speak to its accuracy other than a "works for me". Documentation says it "uses data from the US Census and OpenStreetMap." VMs available. Oh, I see @spara beat me to it :-)

waldoj commented 10 years ago

Y'all just blew my mind.

waldoj commented 10 years ago

Man, this whole thing is just excellent, well beyond geocoding. I can't believe I didn't know about this.

feomike commented 10 years ago

nice one @spara i didn't know about that one either. can you tell me more about the origin?

spara commented 10 years ago

Pete Warden took the geocoder that Schuyler Earle wrote for Fortius/GeoCommons and wrapped a bunch of tools with it. It was sort of an in between project when he was between leaving Apple, playing with big data, and starting JetPac. I think he's added British post codes as well. Pete still maintains it from time to time but he's been focused on JetPac for the past couple of years. I used it in production when I was doing stuff for InfoChimps in 2011 and it seemed pretty stable back them. Just for the sake of disclosure, I do consulting work for JetPac.

spara commented 10 years ago

The grand daddy of online TIGER based geocoders, geocoder.us is still available and has been updated (in 2013). I think Rich Gibson has also added a couple features such as including census information from a geocoding request. I do know that this is currently maintained.

feomike commented 10 years ago

:+1:

spara commented 10 years ago

PostGIS 2.x includes a TIGER based geocoder. It also includes address normalization and the option to offset address points from the street centerline.

petewarden commented 10 years ago

Thanks for the kind words, and USODI looks awesome! The background to the DSTK was my need for a turnkey geocoding solution that included data. There's other amazing open source geocoders out there, I'd call out Nominatim too in addition to the previous ones mentioned, the tricky part is usually populating them.

waldoj commented 9 years ago

The U.S. Census now provides a geocoder. In terms of API UX, it's really quite nice—no auth, it's fast, the results are clean and straightforward. I'm using it at this moment to geocode several hundred thousand business addresses. Unfortunately, it is of course based on TIGER data, so the accuracy isn't great, it does poorly in rural areas, etc. Still, they're doing something great with the data that they have.

hampelm commented 9 years ago

Here's a search of all the ArcGIS online locator services -- aka geocoders -- and a link to the docs via @ajturner

waldoj commented 9 years ago

Here's New York (state's) geocoder, which they promote as publicly available.

waldoj commented 9 years ago

Utah has a geocoder, complete with sample Python. Unfortunately, it requires registration. I don't see any reason to think that registration is limited to any particular group (e.g., state employees).

waldoj commented 9 years ago

Arkansas seems to have a geocoder, but I can't find any documentation about permissible uses. I've linked to sample query, which works great. Nothing on that page says anything about a ToS, copyright, etc.

steveoh commented 9 years ago

I would not recommend using that python script for utah's locator. I would recommend the Geocoding Samples

waldoj commented 9 years ago

Thank you, @steveoh!

waldoj commented 9 years ago

@geostor writes on Twitter that Arkansas' "geocoding service is completely 'feeless' and open to the public, further explaining that they "have not technically stated CC0 (working on this), it would fall under that category. Public, private: have at it!"

feomike commented 9 years ago

fwiw @waldoj Utah, NY and Arkansas are the three states with the top Geospatial leaders in state government. they are all the early adopters. follow the who ...

waldoj commented 9 years ago

Washington DC has a geocoder (part of a suite of RESTful GIS tools), but I can't find any indication as to what its licensing status is. That said, it seems to be well documented, and my guess is that information is published within their "Master Address Repository" documentation.

waldoj commented 9 years ago

Thanks for that, @feomike!

feomike commented 9 years ago

:+1: DC was number 4; he left to become MD GIO. he is in the process of building one. not sure if it is up yet.

ajturner commented 9 years ago

the DSTK geocoder as @spara pointed out is based on the GeoCommons Geocoder and is open-source. It was designed to have modular components for String address tokenization and for data sources. That way it could be easily extended for different addressing schemes and the OSM or other data as they became available.

It's been used since creation for unlimited GeoCommons geocoding and still churning along pretty well, through subject to the quality of Tiger. So I'm interested in how it could be connected to address point services for loading up 'official' and updating address data.