Make "Attach to the roads" feature don't use Routing algorithms and simply attach to the closest roads

[dolfff]

OsmAnd version 4.0.8 (Android).

When navigating an imported GPX track, OsmAnd offers the awesome (and very underexposed) "Attach to the roads" feature. (Thanks to #3908). Here you find a slider setting "Threshold distance" which sets the distance within which OsmAnd tries to find suitable roads.

Problem description:

The algorithm sometimes chooses roads that deviate from your plan. This happens because OsmAnd uses normal intelligent routing rules. So if e.g. the algorithm finds the closest road / path to be not permitted for whatever reason (e.g. wrong vehicle type, restrictions), it calculates a detour if possible within the "Threshold distance". However, this is undesirable in many cases as I 'll explain in the "Proposed solution" paragraph.

The problem can be minimized by using a smaller setting of "Threshold distance", however, that doesn't solve it completely. Even at the smallest setting of 5m, the problem occurs very incidentally. (I need to emphasize that my GPX tracks are very accurate and based on OSM). Unfortunately, it can only be set in steps of 5m. At 0m, road attachment seems to fail completely: no more streetnames and lots of unnecessary navigation instructions (although not as bad as navigating the dumb GPX line as I used to do before this awesome feature was implemented). This makes sense because a GPX track will seldom be exactly upon a road. Nevertheless, this behavior is not very useful and thereby unwanted.

Considerations:

At first I considered that being able to set "Threshold distance" between 1m and 5m could solve the problem. However, that's probably not the case. When this setting is very small, the 0m problem described above will probably occur often. Small errors in the GPX track or map will always exist so a reasonable setting of "Threshold distance" is always required.

It all comes down to the way OsmAnd attaches the line to the roads. It uses normal rules for route calculations based on the chosen navigation-type (vehicle) and road restrictions etc. So, the described behavior is not a bug. But in my opinion, this behavior is unwanted in many cases when using an imported GPX file. This is why: People who use this feature know what they are doing (generally spoken). They use some external tool with it's own intelligence. So, in general, the GPX track is a very meticulously and willingly created track. If the track uses some illegal roads or paths, the creator has its reasons. Perhaps that's why he or she uses the external GPX creation tool in the first place! (Like myself). Any detours are completely unwanted in this case. Using imported GPX files is a very different use case than using OsmAnd's own routing features.

In short: the intelligence is in the GPX track itself. It's preferred that the feature "Attach to the roads" uses a dumb algorithm in these cases, ignoring vehicle type and other road restrictions. Just attach to the road / path that is closest to the line and within the threshold distance. When there is none at all within the threshold distance, use the original line (attachment failed for this segment). (I think the latter is already the case as far as I understand).

When a building architect hands its intelligently created drawings over to the contractor, he wants him to use his intelligence for choosing the right tools, people, planning and so on, NOT FOR ALTERING HIS DRAWINGS, right?

I suppose the "dumb" algorithm as described above, is not always applicable so it should be optional.

Proposed solution:

A:

In the "Attach to the roads" feature, add an option to use a dumb algorithm that forces it to always choose the roads / paths that are closest to the GPX track, even if that violates the normal routing rules.

For example by creating a new option in the same screen where "Threshold distance" is. Something like this:

Use intelligent routing where possible? Toggle: Yes/No At "Yes" add description: "OsmAnd uses normal routing rules for your way of travel. This can cause deviations from your original GPX track if OsmAnd encounters unallowed roads. These deviations can never exceed Distance threshold though." At "No" add description: "OsmAnd disregards all routing rules and will always try to attach to the roads that are nearest to the GPX track (and within Distance threshold). Even if that violates restrictions." (I think "No" should be the default).

B:

I suggest you let the "Threshold distance" start at 1m, not at 0m because that doesn't make much sense.

C:

I suggest to make "Threshold distance" an editable field so as to allow for more precise values.

[vshcherb]

Indeed, we had that idea in mind and I think that how our old algorithm worked 1) it was producing lots of errors when GPX is slightly different than OSM 2) It's very slow (here we calculate routing and in most of the cases it matches perfectly).

I think the problem is quite niche unless we see lots of messages here.

I would propose to create a permissive routing.xml that would accept all possible roads. It's pretty easy to do with custom routing.xml.

P.S.: the dumb algorithm has lots of complications not only with perfect geometry but also with points positioning, super-dumb algorithm would assume you have exact OSM-nodes as GPX-points but I doubt that is any regular use case for that. So in the end dumb algorithm 1) is very different from original 2) still pretty complicated to implement.

[dolfff]

My software teacher used to say: if it's complicated to do something simple, you messed up your design....

I don't think that people who experience problems with the smart routing will come to this thread. It could be a problem that many people experience without understanding what is going on. That's why I emphasized the use case above. In short: using imported GPX means: it's already intelligently worked out and deviations are not desired (mostly), not even when the track is on illegal roads for the chosen vehicle. I can't see why this is "niche". To investigate we should issue a poll somewhere:

If you import gpx that contains roads that are illegal for your vehicle, what do you prefer the "attach to the roads" feature will do? A: stick to the track B: try to divert onto legal roads.

Anyway, I will try to understand and customize routing.xml. Doesn't look simple at first sight though. I have to learn al lot about terminology first.

Apart from my main suggestion A, suggestions B and C still stand, right? Perhaps they deserve a separate issue?

[vshcherb]

I'm not so sure, we will need to collect more feedback. It's not clear how many people would need to have feature "Attach to roads" immediately available

[barteis]

I think what @dolfff is saying is that OsmAnd now has a working "Attach to the roads" feature when following a GPX. In my initial testing, it mostly works well, giving turn-by-turn instructions to an imported GPX created elsewhere. That wasn't possible before, and I think many OsmAnd users will welcome it.

His request has to do with routing. When he imports a GPX, he's wants--as an option--the resulting route to follow that GPX as much as possible, regardless of the rules of the road. This "dumb algorithm" would ignore "vehicle type and other road restrictions. Just attach to the road / path that is closest to the line and within the threshold distance."

[dolfff]

Thanks Bart for clarification. I couldn't have said it better. At first I thought @vshcherb understood what I meant but the latest reaction caused a big question mark floating above my head. "Attach to the roads" feature itself already exists for some time now, see #3908 . So that's not the point.

That wasn't possible before

Well, yes, it WAS possible before but it resulted in lots of unnecessary "turn slightly" instructions. It wasn't map aware so it had to call all the bends in the track. Also couldn't tell streetnames. So "Attach to the roads" is a very big improvement.

And again: if a track is on a forbidden road, it is deliberately there. And if it's not, one should correct the gpx track, not let OsmAnd apply it's own routing rules.

I can't grasp why this would be complicated to create. I have to admit I don't know the code of OsmAnd but I have been coding 25 years of my life so I am not completely clueless. Ignoring some rules doesn't sound complicated to me. Perhaps it's better to create an imaginary super-vehicle that is allowed everywhere. Probably that's what happens if I create my own routing.xml. But still, I can't see why anyone would want OsmAnd to divert from an imported track. If such a "super-vehicle" is easy with routing.xml, then it's also easy in code I presume.

[vshcherb]

@barteis Yes I've reacted to it in my first comment, as it's completely separate function / algorithm. We don't have it implementation. It sounds much more complicated then just "dumb" algorithm cause it could over private roads / rivers / railways, so it needs to be implemented first.

2. It needs to be explained to the user.

So I see it could be a nice extension possibly though it requires a different approach. For now navigation by GPX without roads attach could be used as a workaround.

And to be honest this approach will anyway require list of "possible roads to be attached" without this list turns / roundabouts / max speed won't be calculated correctly.

[dolfff]

Me calling it "dumb algoritm" is just me being bad at English. In my later posts I explained it better and also argue why my suggested approach should be default (why would anyone want OsmAnd to divert from the GPX track?). I really can't see why this is "niche".

I can understand that "dumb" is really not the way to go. But how about my "imaginary super-vehicle" approach (explained in my previous post)? This can be done easily with a custom routing.xml as you explained earlier. So it can also be part of the implementation. Possibly nothing has to change in the routing algorithm itself. OsmAnd just has to acknowledge that a GPX track is ALREADY intelligently routed (in another application). So it can use the imaginary super-vehicle that has hardly any restrictions.

Even if changing UI for creating an option is complicated, I would argue that my suggestion is the default way to go.

[barteis]

@dolfff--with all due respect, I think @vshcherb has a point. The presumption in using any GPS app is that the route created by that app is legal. As a hiker, I don't expect to be taken on freeways. As a driver, I don't expect to be directed to footpaths or the wrong way on one-way streets. A route that does otherwise could be hazardous to the health of unsuspecting users.

When you import a GPX into OsmAnd and follow it, you don't get turn-by-turn directions, but you are following the route exactly as specified, because the underlying OpenStreetMap data isn't used. But once you attach the GPX to the OSM roads and use OsmAnd algorithms to navigate it, this becomes an OsmAnd-generated route like any other, whether based on a GPX or not. That, I think, is what most users will expect.

You could be right that it's worth implementing anyhow, complete with red flags and disclaimers. But I for one have yet to read about any demand for it beyond this discussion.

[dolfff]

Well, I haven't read a lot about the "Attach to the roads" feature as such. Possibly not a lot of people have discovered it? So it's difficult to know what people want or expect from "Attach to the roads" feature at this point. I expect it to attach to the roads only ;)

That being said, my presumption of the use case can be wrong of course. But I am not giving up. Yet..... ;)

The presumption in using any GPS app is that the route created by that app is legal.

This is true of course when the app is calculating a route from point A to B. But is it also true when importing a GPX? I understand that OsmAnd uses route calculation for this feature but in essence the user selected "Attach to the roads". Not "calculate route closest to track". Nothing suggests that route calculation is used or needed. And in fact, it isn't needed indeed. In essence "Attach to the roads" is something different. A complete separate algorithm would have been possible and justifiable. but I presume that would have been too much work. So it was decided to use (parts of) the existing routing algorithm.

So, in short, your presumption is right but my point is that "Attach to the roads" is in essence NOT the same as routing, despite the fact that OsmAnd USES routing algorithm in the background. The routing IS ALREADY done in the other application with which the GPX was created. With it's own prerequisites and perhaps painstakingly alterations by the user. You don't want routing upon routing, do you? You don't want your labor to be undone by another algoritm. The GPX was intelligently created so it can be considered "ready". The only thing that is needed is OsmAnd correlating the track with the map, not any more routing. (But as suggested, if optional, both possibilities exist and can and should be provided with clear descriptions).

When you import a GPX into OsmAnd and follow it, you don't get turn-by-turn directions

Okay, strictly spoken you are right. Let's call it bend-by-bend navigation then. I have used it that way for years, including voice instructions. Be aware that in this way, you can and will be sent on those illegal roads. Without red flags or disclaimers. So my suggested use case was and is already there in OsmAnd. But by using "Attach to the roads" as it is currently implemented, you suddenly change this use case, which, ideally, should not happen. I don't want the use case changed, I just want it to be map-aware, so as to avoid unnecessary or even missing instructions, and tell streetnames.

I would like to put it in another way. I think we should distinguish 3 methods when navigating a GPX track.

1. Only track aware: dumb bend-by-bend navigation. Causing lots of unnecessary "turn slightly" instructions. No streetnames.
2. Map aware. Track has been correlated to the nearest road. No detours. Simple turn-by-turn navigation is possible (at least in theory). Including streetnames.
3. Route aware. Track has been correlated to the nearest legal route. This can cause detours if track is on illegal roads for current way of transport. Full knowledge of all traffic and vehicle rules.

Methods 1 and 2 are essentially the same use case (follow gpx track as close as possible), number 3 is a different use case (make corrections on gpx track to make it legal). So,,,,, I am missing option 2. Desparately. The "pure" implementation of this is cumbersome I suppose. So I suggest OsmAnd keeps on using the algorithm as it is but uses an internal version of routing.xml that allows pretty much any road. So as to respect the intelligence that is IN the GPX itself.

[vshcherb]

I would correct the issue title and label

[dolfff]

Yes, much better title indeed, thanks.

In implementation though, it could still use routing algorithm but use an internal routing.xml of a "super-vehicle" that is allowed on every road. This could drastically simplify implementation.

[bananu7]

I'd just like to say that given the amount of time the app takes even for relatively simple tracks, this would be a welcome addition.

[punyidea]

I would like to say that I would appreciate this feature as well. Specific use case: routing already performed by brouter, I want to check the road surface along that route. To do that, the route has to be fit to roads.

[ems299]

Another vote for this feature.

Use case: Planning a Route on the phone with OSMAnd and BRouter, then navigating this route with OSMAnd.

If I plan a route and have let‘s say for simplicity only a start and an end point. Then I use the BRouter profile to plan the route. To be able to navigate it I‘ll have to attach it to the roads using the builtin bicycle profile. This will, however, result in a completely different route because between each point (in this case start and end) the routing algorithm of the profile is applied.

The only workaround I could find so far is to export the planned route as GPX, open the editor, delete the section at the end of the section. Then import the file again (then a lot of intermediate points are there), attach it to the roads with the bicycle profile and navigate.

If there was a function to add intermediate points to a track at a specified interval (what is done during GPX export), this would solve it as well. But I couldn‘t find this function. It is obviously there, but only in the background for GPX export and has no interface to use it with planned tracks.

[vshcherb]

It's implemented and enabled by default as attach to roads algorithm. Configuration is available in Navigation Settings (with Developer Plugin)