Review how we are using geolocation services.

[ColmMassey]

Identify and try some other services for quality differences

[sunnydean]

Previous Discussions: https://github.com/SolidarityEconomyAssociation/open-data-and-maps-outreach/issues/213 https://github.com/SolidarityEconomyAssociation/open-data-and-maps/issues/113

Current list of potential geocoders and all relevant info will be kept at: Geocoder folder in the nextcloud

[ColmMassey]

The errors here https://github.com/SolidarityEconomyAssociation/open-data/issues/23 could be used as test data.

[sunnydean]

@ColmMassey So, I have almost finalised the research, just waiting for some email responses.

Context: The first geocoder we used was csv2geo and the reason for switching was that we needed to automate the geocoding process, but the service did not allow an API at a reasonable price (only option was 99$/month). We then switched to https://opencagedata.com/ and that was relatively sufficient for a time, but it became obvious that the service was not as accurate as we expected. (This was missed due to insufficient testing)

To solve this issue: In a nutshell 22 geocoding services were informally reviewed in terms of: API easy of use, Policy (in terms of commercial use, caching and values), Price (with regards to our needs), accuracy and coverage (i.e. do they cover regions across the globe and different languages). Informal notes and emails can be found at the Geocoder folder in the Nextcloud.

For accuracy the geocoders were compared to the google maps geolocator (which we assume is the most accurate service currently available) with test cases described in the geocoding document.

Three services were found to fit the requirements of SEA and have an accuracy comparable to google maps: [a] https://csv2geo.com [b] https://locationiq.com/ [c] https://www.geoapify.com/

a,b and c did not initially have a package which would suit the current needs of SEA and were contacted with a brief email describing our needs. [a] dropped their package cost from 99$/m to 49$/m and is currently the service i recommend we use. The way I suggest we use the service is to only pay for the months where we actually need some geocoding done and not pay for months where none is needed. we are still waiting for a response from b and c.

-- Note: I contacted them on behalf of myself and SEA was not mentioned in any way

These three services are easily integratable with the sausage factory (i.e. it would take less than one or two hours to replace the geocage service) The three services above are a scalable solution for both our current and future needs. The idea was to find the most reliable geocoding services that the SEA-mapping project could stick to in the future.

[sunnydean]

Right, with the responses from the emails it comes down to either: csv2geo or geoappify. CSV2GEO is at a price of 49$/month minimum with no attribution required. We can geocode 42600 initiatives with that and then every other initiative we geocode will be priced at $0.00115

Geoappify can be free (but will require attributions to be made). Using the free package we can geocode about 100,000 initiatives per month. We can also pay for the basic version ($53/month) where we get 200,000 geocoding requests and do not have to pay any attributions.

In terms of accuracy both services are comparable to google maps. csv2geo seems to be only slightly more accurate, but nothing significant

in terms of what type of data we get back, for csv2geo we only get only the lat/lng fields. With geoapify we get some corresponding osm data for our requests as well, such as:

type: "Feature" properties: datasource: sourcename: "openstreetmap" neighbourhood: "Bairro Ledidere" hamlet: "Masau de Baixo" oneway: "no" osm_type: "W" osm_id: 24240159 display_name: "Rua 30 de Agosto, Bairro Ledidere, Masau de Baixo, Dili, 10, East Timor" category: "highway" type: "tertiary" name: "Rua 30 de Agosto" old_name: "Av. Pres. Nicolau Lobato" street: "Rua 30 de Agosto" city: "Dili" state: "Dili" postcode: "10" country: "East Timor" country_code: "tl" lon: 125.5879819 lat: -8.5513636 address_level: "street" formatted: "Rua 30 de Agosto, Dili, East Timor" rank: importance: 0.09999999999999998 popularity: 0.5371141775536201 bbox: 0: 125.5857803 1: -8.5519557 2: 125.5900363 3: -8.5508079 geometry: type: "Point" coordinates: 0: 125.5879819 1: -8.5513636

It is really similar (or if not the same) as the data we get from geocage (our current geocoding service). With geopify we also get some other services we might end up using in the future (routing, isolines ...)

I'd say we use geoapify for now and if it doesn't work out go for csv2geo.

Let us schedule a meeting next week to quickly discuss. The switch will be quick :)

[ColmMassey]

This is very thorough. Happy to follow your recommendation. Do you need me to set up an account?