Deal with unnamed OSM points

[vreixo]

When legs have origin or destination on unnamed points, I suppose that when they are not part of a road network, they are returned from OTP as osm:node:identifier.

I don't know if this has some relation with server version or configuration because in Tampa it does I couldn't reproduce it and in A Coruña and The Netherlands it happens all the time. Also trips bicycle are very "discomposed" in A Coruña with multiple sections, I don't know if this might be related.

If this it's something that we should take care client side, would be an option to trigger a geocoding process for the coordinates of that points.

[barbeau]

IIRC the way renaming for pedestrian paths is configurable via OTP at graph building time via custom naming strategies. I remember setting this up in Tampa. Broken bike paths like have to do with bikeable graph connectivity, possibly some OSM data are labeled so they are not bike friendly/routable.

We should probably do renaming in the client for origin and destinations on servers that do not handle this themselves.

[vreixo]

@barbeau I'm working on a new total solution for this.

• The general process:
  1. When a street name is "osm:node" coordinates are displayed instead.
  2. A process for retrieving an address for that point (we know the coordinates) is launched.
  3. If a list of address are obtained for that point, we keep the first one (I suppose that the order returning the results is not casual) that is closer to the original point than a certain amount (the same constant which is used to geocode markers position).
  4. With selected address coordinates are replaced from Marker title or step by step instruction.
  5. If was an step by step instruction adapter gets notified so contents are updated on the screen (for a marker this step is not necessary).
  6. Results are saved to the received itineraries in order to don't need to query again this data from the geocoder server when the app is restarted or removed from cache (really is also restarted).
• Now step by step instructions are generated when itineraries are received instead of creating and re-creating them each time that screen is shown. This way previous process is easier and I think that it is even an improvement.
• I have extended the process to replace all the auto generated location like "road", "path",... by retrieved addresses. Now they are totally replaced but we might consider something like "Road close to [geocoding result]" or even "Road [geocoding result]" but this is dangerous because the geocoding result might be something different close to this.
• I think that now result is much better, specially for that "osm:node:" points where no other information was available apart from coordinates. I always though that "road", "path" and this kind of indications where very vague.
• One possible option with osm:node is to use it for geocoding, but we would need to know which type of feature it is: node, way or relation. One solution will be to make the three queries, but this off course this is only valid if Nominatim is selected (can we ignore the property in this cases?).
• Related with this new mechanism there is a very big problem, too much queries. In the app performance is not an issue at all, and it's very difficult to the user to perceive the geocoding changes, the only way will be to open very quickly step by step directions and he might discover that some results are going to change, but all occurs pretty fast and will be the only time for this request that something will be changed. So, too much queries is a real issue because Google Places API limit is quite easy to reach, I'm seeing now the error "You have exceeded your daily request quota for this API." and is true that I was developing this and testing it a lot in the last 24 hours and also with worse mechanisms that made even more queries, but I'm a little worried about this point. One sideway effect is that the Android Geocoder is also affected and it returns very bad results, and usually more than one (so the distance limit to avoid bad results does not work, one option will be to change this and only offer to the user geocoding results for a marker if they are close enough to that point or even autoselect the closest one (if any) and display near to message). The conclusion is that if the limit is reached and the user has not disactivated "Prefer Android geocoder" he won't be able to plan a trip.
• As a side note, I think that the first cause of "osm:node" are the crosswalks that are used quite often in bicycle routing and don't have any references to the streed (maybe they are incorrectly tagged, I'm not sure).

As you can see there are several problems but I think that this addition could be a very interesting addition. I just wanted to write all the problems, later might be better to try to discompose these in several issues: Google Places API limit, problems when reverse geocoding for marker and several results are obtained, "osm:node" for crosswalks and my comments about the proposed mechanism to see if you agree.

[vreixo]

Well, it seems that the issue might be too many requests per second or something similar, I hope that this was during development and the final solution is not also "abusive". And probably Android grocoder is really disconnected from Google Places, in fact previous issue of Google Places was not affecting the system. http://stackoverflow.com/questions/8218764/android-geocoder-quota-limits

[vreixo]

@barbeau I'm trying with another device with the play store version installed and still is not working Android Geocoder, last option is that my IP is being restricted, but I can't use my data plan on that device.

I'm afraid that our app is banned in general for Android Geocoder, but the problem is that instead of not giving any result it does return bad results not letting the fallback geocoders Google Places or Nominatim give proper results. If this is confirmed I'm wondering if this will be just temporary or we should do something to recover the functionality.

@kalon33 do you have this issue too?

[kalon33]

@vreixo: what is the problem and how to test it (sorry, I'm writing my thesis and analysing results right now, so my overall comprehension of the world is messed up ^^)? If you wanna say that Android Geocoder is not working anymore, it also don't seem to work right now on my phone, but it was properly working yesterday or the day before (as I do always the same trip without completely closing the app, I don't have to use geocoding each time).

[vreixo]

@kalon33 I'm writting my "thesis" too, what a coincidence :-) (I'm not sure how to call it in English, it's just the final project after a five years university)

Yes my point was if Android Geocoder for the app will be working since this morning in other phones or only my phone was banned. Not very good news :-(, I hope that this is just temporary.

2014-08-27 18:01 GMT+02:00 Nicolas Derive notifications@github.com:

@vreixo https://github.com/vreixo: what is the problem and how to test it (sorry, I'm writing my thesis and analysing results right now, so my overall comprehension of the world is messed up ^^)? If you wanna say that Android Geocoder is not working anymore, it also don't seem to work right now on my phone, but it was properly working yesterday or the day before (as I do always the same trip without completely closing the app, I don't have to use geocoding each time).

— Reply to this email directly or view it on GitHub https://github.com/CUTR-at-USF/OpenTripPlanner-for-Android/issues/357#issuecomment-53596658 .

[barbeau]

@vreixo Looks like we've hit our 1k requests/day Places API quota a few times:

I would imagine that the Android geocoder is connected to Places API behind-the-scenes, but we don't have any way of knowing that, and its definitely not tied to our Places API quota. But yes, they might ban perceived abuse of the Android Geocoder as well. Geocoding doesn't seem to be working for me at present, either (If I search for "Burger King", all I get is "Tampa, FL, Pinellas County, FL" and a few more geographic areas. So yes, looks like we need to be careful here - hopefully this is just the daily quota limiting access.

How can I tell if the results are coming from the Android geocoder and not the Places API?

Also, when you say - Replaced osm:node in street names by coordinates, do you mean replacing it with latitude and longitude? I'd prefer to present generic names such as "sidewalk" or "street" rather than raw lat/long. Could you provide examples so I can understand what you mean better?

[vreixo]

@barbeau ups, I'm sorry for breaking the geocoder for everyone, I have thought that any limits will be linked with the device or the IP, and not ban everyone. Let's hope that this is temporary and, specially Android Geocoder, will work soon again.

I'm attaching some screenshots to clarify how master deals with the unnamed ways and how this is very problematic for bike trips in my city. I was investigating OTP configuration but I didn't found a solution. It's a shame to have reached the data usage limits because with the solution of geocode this results all the streets were retrieved and the result was great. The places tagged with name osm:node: don't have any useful data, except for the node number itself (only for Nominatim of course) and the latitude and longitude. So that's way I have used lat/lon, if really all the time are crosswalks we can safely say "Walk to sidewalk" or also "Walk to street" which should be always more or less ok, although it seems a bit weird to me, but at least we don't use coordinates which are not user friendly. You can see an example of "Ride to road" in the last screenshot which does not seem like the best solution. UPDATE Btw we also need to change that destination, because there is a lat/lon there, and this is the code from master branch without my modifications. ->I was using non intelligent markers, so this last point is not a problem :-)

[kalon33]

@vreixo : Did you needed to use the geocoder for only once to fix all these roads (once but for lots of requests) and then the OSM data is fixed, or every time? Because if it's only once, we could imagine create a special Google account for that.

Is there a way to use the geocoder to contribute back the data you retrieve from it to OSM or not? In order to fix data once for all? (maybe I don't really understand what you're doing with that data)

----- Mail original -----

De: "Vreixo González Caneda" notifications@github.com À: "CUTR-at-USF/OpenTripPlanner-for-Android" OpenTripPlanner-for-Android@noreply.github.com Cc: "Nicolas Derive" kalon33@ubuntu.com Envoyé: Mercredi 27 Août 2014 22:12:09 Objet: Re: [OpenTripPlanner-for-Android] Deal with unnamed OSM points (#357)

@barbeau ups, I'm sorry for breaking the geocoder for everyone, I have thought that any limits will be linked with the device or the IP, and not ban everyone. Let's hope that this is temporary and, specially Android Geocoder, will work soon again.

I'm attaching some screenshots to clarify how master deals with the unnamed ways and how this is very problematic for bike trips in my city. I was investigating OTP configuration but I didn't found a solution. It's a shame to have reached the data usage limits because with the solution of geocode this results all the streets were retrieved and the result was great. The places tagged with name osm:node: don't have any useful data, except for the node number itself (only for Nominatim of course) and the latitude and longitude. So that's way I have used lat/lon, if really all the time are crosswalks we can safely say "Walk to sidewalk" or also "Walk to street" which should be always more or less ok, although it seems a bit weird to me, but at least we don't use coordinates which are not user friendly. You can see an example of "Ride to road" in the last screenshot which does not seem like the best solution. Btw we also need to change that destination, because there is a lat/lon there, and this is the code from master branch without my modifications.

— Reply to this email directly or view it on GitHub .

[barbeau]

@vreixo No worries, hopefully everything is fine tomorrow :).

Here are my thoughts for displaying osm:node vs. coordinates vs. place name:

A. For entire trip origin/destination, we should always try to find the place name associated with the location and show that to the user B. For itinerary steps and mode changes, I would prefer to use generic description of whatever feature it is (crosswalk/road/sidewalk). I would prefer not to show lat/long or osm node IDs, since neither will be useful to most users.

For A, from my experience we are very likely to get a landmark back that is meaningful to the user that is roughly near their intended destination.

For B, the reason why I don't prefer a place name, is that in my experience it wouldn't result in a useful name in most cases (at least from the data I've seen). I think you're likely to get confusing results with a landmark that is somewhat far away from the OSM way - and, for step-by-step directions within a trip, I'd rather give less precise information that is more accurate, than more precise information that is less accurate - if that makes sense.

@vreixo is there a way you can tell how far the place name is from the lat/long of the node that you reverse geocoded?

Having said the above, I'm interested to see the results of your work. Perhaps I'm wrong on B, and it is possible to match highly precise place names for step-by-step direction steps.

[vreixo]

@barbeau thanks :-)

About #365, I was talking about A. That issue is related to the problems that appear when the user sets origin/location touching the map, now he will get a list of far away results to choose for that point coming from the broken Android Geocoder. We need to fix that for possible new problems with geocoding usage limits.

About the rest of your comment you might be right :-) I just felt very unnatural those messages of "Ride to road" or "Walk to sidewalk", maybe is a problem with the translations and might sound better in English than in my language.

Regarding you question, yes there is. I have the original lat/lon and the geocoding results also have these fields. Although, I don't think that won't be possible to get good results, because even if the new Address is very close it might just be in the crossing street for example and might be more confusing, as you have said.

If I have new thoughts about this I'll write them here, but now I think that I won't continue working on geocoding the intermediate steps and I'll try to focus on how to solve my issues from OTP.

[barbeau]

@vreixo Ok, that sounds good. Its a little awkward in English too, but I think better than the alternatives.