Error in polyline-encoded geometry

[sfendrich]

Here's what I did

curl -X POST   https://api.openrouteservice.org/v2/directions/driving-car \
   -H 'Authorization: <mykey>' \
   -H 'Content-Type: application/json' \
   -d '{"coordinates": [[11.300716,48.197123],[11.449067,48.125142]]}'

---

Here's what I got

Unexpected geometry (see online polyline decoder) in this result:

{
  "routes": [
    {
      "summary": {
        "distance": 25503.5,
        "duration": 1330.1
      },
      "segments": [
        {
          "distance": 25503.5,
          "duration": 1330.1,
          "steps": [
            {
              "distance": 139.6,
              "duration": 100.5,
              "type": 11,
              "instruction": "Head northwest",
              "name": "-",
              "way_points": [
                0,
                1
              ]
            },
            {
              "distance": 148.9,
              "duration": 10.7,
              "type": 3,
              "instruction": "Turn sharp right onto Schloßstraße",
              "name": "Schloßstraße",
              "way_points": [
                1,
                6
              ]
            },
            {
              "distance": 244.5,
              "duration": 27.3,
              "type": 7,
              "instruction": "Enter the roundabout and take the 3rd exit onto St 2345",
              "name": "St 2345",
              "exit_number": 3,
              "way_points": [
                6,
                22
              ]
            },
            {
              "distance": 172.3,
              "duration": 31.6,
              "type": 0,
              "instruction": "Turn left",
              "name": "-",
              "way_points": [
                22,
                34
              ]
            },
            {
              "distance": 5183.2,
              "duration": 226.1,
              "type": 5,
              "instruction": "Turn slight right onto B 471",
              "name": "B 471",
              "way_points": [
                34,
                110
              ]
            },
            {
              "distance": 3771.9,
              "duration": 171.1,
              "type": 13,
              "instruction": "Keep right",
              "name": "-",
              "way_points": [
                110,
                142
              ]
            },
            {
              "distance": 4568.4,
              "duration": 132.5,
              "type": 12,
              "instruction": "Keep left onto A 8",
              "name": "A 8",
              "way_points": [
                142,
                160
              ]
            },
            {
              "distance": 809.2,
              "duration": 36.6,
              "type": 13,
              "instruction": "Keep right",
              "name": "-",
              "way_points": [
                160,
                177
              ]
            },
            {
              "distance": 6256.8,
              "duration": 286.5,
              "type": 12,
              "instruction": "Keep left",
              "name": "-",
              "way_points": [
                177,
                227
              ]
            },
            {
              "distance": 3475.8,
              "duration": 205.1,
              "type": 12,
              "instruction": "Keep left",
              "name": "-",
              "way_points": [
                227,
                282
              ]
            },
            {
              "distance": 236.9,
              "duration": 17.1,
              "type": 13,
              "instruction": "Keep right",
              "name": "-",
              "way_points": [
                282,
                285
              ]
            },
            {
              "distance": 210.6,
              "duration": 31.3,
              "type": 13,
              "instruction": "Keep right",
              "name": "-",
              "way_points": [
                285,
                293
              ]
            },
            {
              "distance": 77.4,
              "duration": 11.1,
              "type": 1,
              "instruction": "Turn right onto Am Haag",
              "name": "Am Haag",
              "way_points": [
                293,
                294
              ]
            },
            {
              "distance": 133.8,
              "duration": 32.1,
              "type": 1,
              "instruction": "Turn right",
              "name": "-",
              "way_points": [
                294,
                301
              ]
            },
            {
              "distance": 74.3,
              "duration": 10.7,
              "type": 1,
              "instruction": "Turn right onto Kleinhaderner Weg",
              "name": "Kleinhaderner Weg",
              "way_points": [
                301,
                305
              ]
            },
            {
              "distance": 0,
              "duration": 0,
              "type": 10,
              "instruction": "Arrive at Kleinhaderner Weg, on the right",
              "name": "-",
              "way_points": [
                305,
                305
              ]
            }
          ]
        }
      ],
      "bbox": [
        11.299312,
        48.124411,
        11.449739,
        48.232265
      ],
      "geometry": "gpdeHif~cAmDnGg@aC[wAAEY}AOo@FIF[AQEMIKIEK@IFGJCHANBTcAb@yA`@{AXa@FER?RBZPbAL`CCTERIROLODMA]KuAuBqByC{@wAgAmB{@aB_AkBeCgF]u@q@yAoEeJmBmDsB_DsBuCy@cAgFmF}@u@sAgAiCcB{CaB}As@_Bo@aAY}A_@uBa@gDs@aDu@yAa@oBs@qAm@wAy@g@[mAy@wAqAwAuAcBqBgBmCyAkCqAoCoAiDkAyD_AeEk@cDi@cEa@oEo@uKa@}Fe@aEm@qDeAsEe@aBY}@_@gA_@aAc@iAq@{AiCiF}AsCc@iAMUy@{Au@qAu@uAy@_BkAaCcCiF}BmF_DyHyHuSkBmEu@wAw@oAg@q@s@o@MMQOECW]Ye@MYCU@WBQVy@t@oAFWJ_@Fc@?y@WiBGw@?e@Fi@DS@U\\k@zA}ChCuFp@sB`DqGpI_QjOwZd_@qu@|Ted@|EwJhKaTxCaGtEiJ~@iBPi@bGwLbDmGhByDvKoTdF}JfCwEpSk^Zk@fNaV~H{M`GeK`LyRnFmJtUma@~Wyd@nDkGnGyKXQlAq@d@Gd@C~ANx@Cb@OvAu@b@Qd@?^Hb@Vf@d@NBjBrBdFvFrG~HhD|D|@jA\\NdBfCpC`EpArBjDhF~AhCpApB^j@|DlGbBxCjK~QdPj[t\\zd@zOzSjHxIhCxCtHpInBrBvBlBlCrBnBlAbB~@xAn@vDlAxA`@hCf@bBVlBPjBFzB@l@Ap@AhF[rFiA~@WhA_@jI_DdCaAjEgBz@_@tEqBvGqCxEuBvQiIvFwChEuBzCeBfBsAfA}AxAiCl@yAh@yBZuBLgBBuBA{ACmAUaCe@wBc@yAwA_Dy@}Bi@sBOc@eAmHo@cGm@cHSiDOkDGsBIyBK_J?wDF{DHeCJmBVkDJgA^uCd@wCj@uCbAoEjFsSv@_Df@oC^mCRaCPuD@uCE_DSiDWmCsAiJScCMyBIwD?oBBsBPoDX}C`@oCVwA`@gBh@eAjBqFvAwELM\\}@Zi@X_@b@QVCd@@fC\\MnEqAIMFClAITSA[CWC?k@BoA@YBq@",
      "way_points": [
        0,
        305
      ],
      "warnings": [
        {
          "code": 1,
          "message": "There may be restrictions on some roads"
        }
      ],
      "extras": {
        "roadaccessrestrictions": {
          "values": [
            [
              0,
              1,
              4
            ],
            [
              1,
              294,
              0
            ],
            [
              294,
              296,
              32
            ],
            [
              296,
              305,
              0
            ]
          ],
          "summary": [
            {
              "value": 0,
              "distance": 25309.8,
              "amount": 99.24
            },
            {
              "value": 4,
              "distance": 139.6,
              "amount": 0.55
            },
            {
              "value": 32,
              "distance": 54.1,
              "amount": 0.21
            }
          ]
        }
      }
    }
  ],
  "bbox": [
    11.299312,
    48.124411,
    11.449739,
    48.232265
  ],
  "metadata": {
    "attribution": "openrouteservice.org | OpenStreetMap contributors",
    "service": "routing",
    "timestamp": 1556185224354,
    "query": {
      "coordinates": [
        [
          11.300716,
          48.197123
        ],
        [
          11.449067,
          48.125142
        ]
      ],
      "profile": "driving-car",
      "format": "json"
    },
    "engine": {
      "version": "5.0",
      "build_date": "2019-02-28T09:09:39Z"
    }
  }
}

---

Here's what I was expecting

The geometry shown in the playground map, when using the same coordinates.

EDIT: Clarification: the issue is that the polyline decoder and the ORS playground show different geometries. The geometry shown in the polyline decoder is wrong, the one shown in the playground is correct.

[nilsnolde]

Please try that again.. Works for me with Google's link and with all my libraries.

[aoles]

I've tried the same API query as @sfendrich and got the same result as he with the following geometry.

gpdeHif~cAmDnGg@aC[wAAEY}AOo@FIF[AQEMIKIEK@IFGJCHANBTcAb@yA`@{AXa@FER?RBZPbAL`CCTERIROLODMA]KuAuBqByC{@wAgAmB{@aB_AkBeCgF]u@q@yAoEeJmBmDsB_DsBuCy@cAgFmF}@u@sAgAiCcB{CaB}As@_Bo@aAY}A_@uBa@gDs@aDu@yAa@oBs@qAm@wAy@g@[mAy@wAqAwAuAcBqBgBmCyAkCqAoCoAiDkAyD_AeEk@cDi@cEa@oEo@uKa@}Fe@aEm@qDeAsEe@aBY}@_@gA_@aAc@iAq@{AiCiF}AsCc@iAMUy@{Au@qAu@uAy@_BkAaCcCiF}BmF_DyHyHuSkBmEu@wAw@oAg@q@s@o@MMQOECW]Ye@MYCU@WBQVy@t@oAFWJ_@Fc@?y@WiBGw@?e@Fi@DS@U\\k@zA}ChCuFp@sB`DqGpI_QjOwZd_@qu@|Ted@|EwJhKaTxCaGtEiJ~@iBPi@bGwLbDmGhByDvKoTdF}JfCwEpSk^Zk@fNaV~H{M`GeK`LyRnFmJtUma@~Wyd@nDkGnGyKXQlAq@d@Gd@C~ANx@Cb@OvAu@b@Qd@?^Hb@Vf@d@NBjBrBdFvFrG~HhD|D|@jA\\NdBfCpC`EpArBjDhF~AhCpApB^j@|DlGbBxCjK~QdPj[t\\zd@zOzSjHxIhCxCtHpInBrBvBlBlCrBnBlAbB~@xAn@vDlAxA`@hCf@bBVlBPjBFzB@l@Ap@AhF[rFiA~@WhA_@jI_DdCaAjEgBz@_@tEqBvGqCxEuBvQiIvFwChEuBzCeBfBsAfA}AxAiCl@yAh@yBZuBLgBBuBA{ACmAUaCe@wBc@yAwA_Dy@}Bi@sBOc@eAmHo@cGm@cHSiDOkDGsBIyBK_J?wDF{DHeCJmBVkDJgA^uCd@wCj@uCbAoEjFsSv@_Df@oC^mCRaCPuD@uCE_DSiDWmCsAiJScCMyBIwD?oBBsBPoDX}C`@oCVwA`@gBh@eAjBqFvAwELM\\}@Zi@X_@b@QVCd@@fC\\MnEqAIMFClAITSA[CWC?k@BoA@YBq@

This when pasted into Google's online polyline decoder results in a geometry different from the expected one shown in the playground.

[TimMcCauley]

Did you actually think about this being a bug of google? @aoles did you decode the line and double check?

[aoles]

Not yet, that's why it needs further investigation. Just confirming @sfendrich's observation. From @nilsnolde comment I've got the impression that he didn't observe the issue even with google's online decoder.

UPDATE I've just tried an alternative mapbox JS implemenation to decode the result and got the same list of coordinates as from Google's online utility.

[nilsnolde]

Oh sorry, I misunderstood indeed. I thought he couldn't decode the polyline at all.

You're right, that'd be seriously awkward.. I'll try to decode and plot real quick here.

EDIT: Aren't we using GH's encoding lib here?

[aoles]

Aren't we using GH's encoding lib here?

Apparently not, it seems that we use our own heigit.ors.util.PolylineEncoder, even though GraphHopper provides com.graphhopper.http.WebHelper.encodePolyline().

[nilsnolde]

Not that I'd have expected a different outcome decoding myself, but yeah, it's wrong.

@aoles do you have time to fix real quick?

@rabidllama I think this qualifies for an immediate hotfix. What do you think?

[nilsnolde]

even though GraphHopper provides com.graphhopper.http.WebHelper.encodePolyline().

:roll_eyes: can we pls use GH? They do the exact same thing, i.e. provide elevation info etc.

[aoles]

OK, I will then look into switching from our home-grown solution to GH's encoder.

UPDATE: it turns out that using GH's encoder is not entirely straightforward. This is so because it lives in graphhopper-web module which we currently don't include. I've tried enabling it but because of cross-dependencies on the rest of the unused modules and our modifications to graphhopper-core I ended up in having to modify a lot of things to enable this single import. Mentioned this to @rabidllama and our conclusion was that this approach doesn't seem feasible. I will rather look into debugging our current code.

[aoles]

There seem to be a similar problem with GH's implementation too, as the queryhttps://graphhopper.com/api/1/route?point=48.197123,11.300716&point=48.125142,11.449067&vehicle=car&debug=true&key=<API-KEY>&type=json returns

gpdeHgf~cAmDlGmBmJFID[?QEMIKICM?IFITANBTcAb@yA`@}Bb@ER@l@PdAL~BGj@IROLQBM?[MiEoGcCeE{BkEcL}UoBoDsB_DsBuCy@aAeFoFsC{BiCeByC_B_Bu@}Am@cAY}OkDwAa@qBu@oAm@wAw@uBwAyAoAuAwAcBqBgBmC{AiCoAqCoAiDkAyD_AcEm@cDi@cEa@oEm@uKa@}Fe@cEm@qDgAqE}@_D_AiCwAeDiCkF{AsCq@}AqBmDmBuDmAaCaCkF_CmF_DyHwHsSkBmEw@yAu@oAg@q@yAqAs@cAKWCU?WBQXy@r@oARy@Fc@@y@YiBGu@?e@Fk@FQ@UvBkEjCsFp@sBjhB{qDnLuUPi@dLgUhByDvKmTfF_KdCuEtc@{v@`I{M~FcKfj@u`A~e@az@fBaAd@If@A~ANx@Eb@MtAw@b@Qd@?`@Hb@Vf@f@L@rIlJrG~HhD|D|@hAZNvFhI|F|InKtPnNxVdPl[t\\xd@|O|SrLpNvHrIlBrBvBlBlCpBnBlAbB~@zAp@tDlAxA^hCh@bBTnBPjBFfD@|G]pFkA`AWtK_ElKiEfUwJtQiIvFyCjEuBzCeBfBsAfA{AvAkCn@yAf@wBZwBLeBDwBGiDUaCc@wBe@wAuAaDy@}Bi@qBQc@eAmHo@eGk@aHSkDOkDQmFK}I?wDD{DHgCJkBVmDj@}Ed@uCj@wCdAmEbHsXd@oC^oCT_CPuD@wCE}CSkDYmCqAiJUaCMyBIyD?oBDqBNoDZ_D^mCVwA`@iBj@cAhBqFvAwELO\\{@Zk@X]b@SVAd@@hDf@TNR\\Hp@StJ_CO_AUgAe@ByCH{C

UPDATE: this gets really weird as we virtually use the official Google implementation.

[TimMcCauley]

@aoles is this the same result for with and without elevation?

[nilsnolde]

Haha wtf.. how can it be that the geometry is correct for a major part of the way and then deviates into complete bullshit.. 3D doesn't explain that. That just completely destroy it. Reeeeally curious what you come up with @aoles!

[aoles]

Found it! It seems to be caused by the escaping of \ in the encoded string, because when I replace all the occurrences of \\ by \ the routes are back to normal.

The escaping happens most probably when creating a valid JSON response object from an originally unescaped string. Not sure this is actually an issue, because according to specification any \ contained in a JSON string should be escaped.

[TimMcCauley]

Great find @aoles - should we add a note to the readme with a link to this issue and close it?

[nilsnolde]

ah damn.. little of a dilemma isn't it.. Wouldn't like to go against JSON specs, but I really think noone should have to manually remove \s, which requires a decoding step. So this should be an exception. Google doesn't escape it either apparently, otherwise their tool would include a decoding for escaping. Is there a way to not escape anything in the geometry object in the first place?

[TimMcCauley]

I think google does, e.g. "overview_polyline" : { "points" : "knjmEnjunUbKCfEA?_@]@kMBeE@qIIoF@wH@eFFk@WOUI_@?u@j@k@`@EXLTZHh@Y`AgApAaCrCUd@cDpDuAtAoApA{YlZiBdBaIhGkFrDeCtBuFxFmIdJmOjPaChDeBlDiAdD}ApGcDxU}@hEmAxD}[tt@yNb\\yBdEqFnJqB~DeFxMgK~VsMr[uKzVoCxEsEtG}BzCkHhKWh@]t@{AxEcClLkCjLi@`CwBfHaEzJuBdEyEhIaBnCiF|K_Oz\\ {MdZwAbDaKbUiB|CgCnDkDbEiE|FqBlDsLdXqQra@kX|m@aF|KcHtLm@pAaE~JcTxh@w\\`v@gQv`@}F`MqK`PeGzIyGfJiG~GeLhLgIpIcE~FsDrHcFfLqDzH{CxEwAbBgC|B}F|DiQzKsbBdeA{k@~\\oc@bWoKjGaEzCoEzEwDxFsUh^wJfOySx[uBnCgCbCoFlDmDvAiCr@eRzDuNxC_EvAiFpCaC|AqGpEwHzFoQnQoTrTqBlCyDnGmCfEmDpDyGzGsIzHuZzYwBpBsC`CqBlAsBbAqCxAoBrAqDdDcNfMgHbHiPtReBtCkD|GqAhBwBzBsG~FoAhAaCbDeBvD_BlEyM``@uBvKiA~DmAlCkA|B}@lBcChHoJnXcB`GoAnIS~CIjFDd]A|QMlD{@jH[vAk@`CoGxRgPzf@aBbHoB~HeMx^eDtJ}BnG{DhJU`@mBzCoCjDaAx@mAnAgCnBmAp@uAj@{Cr@wBPkB@kBSsEW{GV}BEeCWyAWwHs@qH? cIHkDXuDn@mCt@mE`BsH|CyAp@}AdAaAtAy@lBg@pCa@jE]fEcBhRq@pJKlCk@hLFrB@lD_@xCeA`DoBxDaHvM_FzImDzFeCpDeC|CkExDiJrHcBtAkDpDwObVuCpFeCdHoIl\\uBjIuClJsEvMyDbMqAhEoDlJ{C|J}FlZuBfLyDlXwB~QkArG_AnDiAxC{G|OgEdLaE`LkBbEwG~KgHnLoEjGgDxCaC`BuJdFkFtCgCnBuClD_HdMqEzHcBpB_C|BuEzCmPlIuE|B_EtDeBhCgAdCw@rCi@|DSfECrCAdCS~Di@jDYhA_AlC{AxCcL`U{GvM_DjFkBzBsB`BqDhBaEfAsTvEmEr@iCr@qDrAiFnCcEzCaE~D_@JmFdGQDwBvCeErEoD|BcFjC}DbEuD~D`@Zr@h@?d@Wr@}@vAgCbEaHfMqA`Cy@dAg@bAO`@gCi@w@W" },

[aoles]

ah damn.. little of a dilemma isn't it.. Wouldn't like to go against JSON specs, but I really think noone should have to manually remove \s, which requires a decoding step. So this should be an exception. Google doesn't escape it either apparently, otherwise their tool would include a decoding for escaping. Is there a way to not escape anything in the geometry object in the first place?

Actually, I think that we should keep the escaping as is. For example, it is in general necessary for any string literals in R as well and the decoding of the current output works just fine with googlePolylines package.

The same applies when working with openelevation service: the input encoded polyline needs to have the \ escaped as \\ .

In fact, I now believe the problem is not on our side, but rather in the implementation of Google's Interactive Polyline Encoder Utility which does not account for the escaping.

[TimMcCauley]

Yes, agreed. Closing this for now, let’s keep at note on the docs, I’ll take care of this.

[sfendrich]

Agreed. Thanks a lot for debugging! The documentation at /v2/directions/{profile} and /v2/directions/{profile}/json should be extended by something like "The geometry is returned as a string in the Encoded Polyline Algorithm Format. Note that according to the JSON-specification some special characters in this string are backslash-escaped and must be unescaped before decoding."