heremaps / flexible-polyline

Flexible Polyline encoding: a lossy compressed representation of a list of coordinate pairs or triples
MIT License
92 stars 77 forks source link

Off-by-one error in signed integer encoding specification #9

Closed KalleOlaviNiemitalo closed 4 years ago

KalleOlaviNiemitalo commented 4 years ago

In the binary representation of signed integers, the least significant bit stores the sign and the subsequent bits store the absolute number value. So for example, the number 7 is represented as 1110 and -7 as 1111. After this transformation, the normal unsigned integer encoding algorithm can be used.

That claim does not match the reference implementations and produces incorrect results when decoding polylines returned by HERE. The encoding actually goes like this:

lene commented 4 years ago

Thanks for noticing! Fixed in https://github.com/heremaps/flexible-polyline/pull/12.