go-spatial / tegola

Tegola is a Mapbox Vector Tile server written in Go
http://tegola.io/
MIT License
1.29k stars 196 forks source link

Rounding/encoding issue with numeric fields in postgis provider #750

Open russss opened 3 years ago

russss commented 3 years ago

I have been putting off investigating this for a while, and it appears I've found a fix for my use case, but I thought I should report this as an issue.

Consider the following test case:

tegola_rounding_test=# select * from test;
 id |                        geom                        |  output
----+----------------------------------------------------+----------
  1 | 0101000020110F0000014630D227F215C14908E08BD9C15C41 | 0.045080
(1 row)
Full test config Schema: ``` create table test(id serial, geom geometry(Geometry,3857), output decimal(10, 6)); insert into test(id, geom, output) values(1, ST_Transform(ST_SetSRID(ST_GeomFromText('POINT(-3.23 55.9)'), 4326), 3857), 0.045080); ``` config.toml: ``` [[maps]] name = "map" [[maps.layers]] min_zoom = 2 max_zoom = 17 provider_layer = "postgis.test" [[providers]] max_connections = 20 host = "localhost" name = "postgis" database = "tegola_rounding_test" srid = 3857 type = "postgis" port = 5432 user = "russ" password = "" [[providers.layers]] name = "test" id_fieldname="id" geometry_type = "Point" sql = "SELECT id, ST_AsBinary(geom) AS geom, output AS output FROM test WHERE geom && !BBOX!" [webserver] port = ":8081" cors_allowed_origin = "*" ```

When rendered (using the postgis provider), the "output" field shows an incorrect, long repeating fractional part (this is especially annoying because Mapbox/Maplibre GL doesn't have configurable rounding): image

This does not happen with the mvt_postgis provider, so my suspicion is that it's either an issue with Tegola itself or with the golang protobuf library.

When I last looked at this a while back (there is a little more info in openinframap/styles#34), I came to the conclusion that there was a single-bit error in the protobuf encoding of the value. This doesn't happen with all decimal values, just a few of them.

I think my solution to this will be to switch to mvt_postgis, but I thought I should file an issue in case anyone else hits this annoying bug.

ARolek commented 3 years ago

@russss thanks for the report. I agree with your assessment. This is probably an issue in the Postgres driver or the photobuff library. The Postgres driver is due for an update soon, which may address this problem. I appreciate you documenting it.

russss commented 3 years ago

An update to this: it turns out that this does also happen with the mvt_postgis provider.

I thought this was fixed because my columns were of the NUMERIC type in Postgres, and the mvt_postgis provider outputs NUMERIC values as strings in the resulting MVT. (Either the native Tegola MVT support outputs NUMERIC as MVT floats, or I changed the data type around the same time.)

So it seems unlikely this is a Tegola bug after all, although that does make me wonder what is causing it...

ARolek commented 3 years ago

@russss are the results the same with mvt_postgis then? It's strange (and annoying) to have the precision getting messed up like this.