pq: Geohash requires inputs in decimal degrees.

[torse]

Importing with -optimize suddenly stops with pq: Geohash requires inputs in decimal degrees. Here are the last log statements from imposm3 (0.2.0dev-20160311):

... [May 8 23:45:27] [INFO] Importing OSM data took: 55h9m46.845149953s [May 8 23:45:32] [INFO] [PostGIS] Indexing osm_aeroways on geohash took: 4.26745057s [May 8 23:45:38] [INFO] [PostGIS] Indexing osm_amenities on geohash took: 10.206451727s [May 8 23:45:38] [INFO] [PostGIS] Clustering osm_aeroways on geohash took: 6.028841804s [May 8 23:45:39] [INFO] [PostGIS] Analysing osm_aeroways took: 551.59263ms [May 8 23:45:48] [INFO] [PostGIS] Indexing osm_barrierpoints on geohash took: 19.959161366s [May 8 23:45:49] [INFO] [PostGIS] Indexing osm_transport_points on geohash took: 20.683308098s [May 8 23:45:49] [INFO] [PostGIS] Clustering on geometry took: 21.260772732s [May 8 23:45:49] pq: Geohash requires inputs in decimal degrees.

Is this a bug or does anyone know a workaround?

[olt]

Can you post your mapping and the full imposm command?

On 10.05.2016, at 09:16, torse notifications@github.com wrote:

Importing with -optimize suddenly stops with pq: Geohash requires inputs in decimal degrees. Here are the last log statements from imposm3 (0.2.0dev-20160311):

... [May 8 23:45:27] [INFO] Importing OSM data took: 55h9m46.845149953s [May 8 23:45:32] [INFO] [PostGIS] Indexing osm_aeroways on geohash took: 4.26745057s [May 8 23:45:38] [INFO] [PostGIS] Indexing osm_amenities on geohash took: 10.206451727s [May 8 23:45:38] [INFO] [PostGIS] Clustering osm_aeroways on geohash took: 6.028841804s [May 8 23:45:39] [INFO] [PostGIS] Analysing osm_aeroways took: 551.59263ms [May 8 23:45:48] [INFO] [PostGIS] Indexing osm_barrierpoints on geohash took: 19.959161366s [May 8 23:45:49] [INFO] [PostGIS] Indexing osm_transport_points on geohash took: 20.683308098s [May 8 23:45:49] [INFO] [PostGIS] Clustering on geometry took: 21.260772732s [May 8 23:45:49] pq: Geohash requires inputs in decimal degrees.

Is this a bug or does anyone know a workaround?

— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub

[torse]

Mapping: http://pastebin.com/3YxYgfqV

Command: imposm3 import -srid=4326 -mapping=etc/imposm/mapping.json -read=data/planet-latest.osm.pbf -write=true -connection=postgis://<user>@<host>:5434/osm -cachedir=data/cache -overwritecache -optimize=true -diff=true

The planet-latest file is from 2016-04-27.

[torse]

Not sure if relevant, but I there were three GEOS (3.5.0) topology exceptions during import:

[May 6 18:05:56] [INFO] [GEOS] TopologyException: side location conflict at 18.659793725480256 44.570205227300761 [May 6 18:24:47] [INFO] [GEOS] TopologyException: side location conflict at -74.521022712172766 48.374912932309172 [May 6 19:43:53] [INFO] [GEOS] TopologyException: side location conflict at 126.73384511620145 63.452010798820623

[torse]

Upgraded to Postgresql 9.5.2 (was on 9.4.6) and now got bit more information out of the same error:

... [May 15 06:00:39] [INFO] [PostGIS] Creating geometry indices took: 2h2m16.138478147s [May 15 06:00:39] [INFO] Importing OSM data took: 35h21m10.514830309s [May 15 06:00:54] [INFO] [PostGIS] Indexing osm_aeroways on geohash took: 14.565998156s [May 15 06:01:03] [INFO] [PostGIS] Indexing osm_amenities on geohash took: 23.117794915s [May 15 06:01:09] [INFO] [PostGIS] Clustering osm_aeroways on geohash took: 15.184601233s [May 15 06:01:11] [INFO] [PostGIS] Analysing osm_aeroways took: 1.409130871s [May 15 06:01:31] [INFO] [PostGIS] Clustering osm_amenities on geohash took: 28.568787914s [May 15 06:01:32] [INFO] [PostGIS] Analysing osm_amenities took: 270.367341ms [May 15 06:02:11] [INFO] [PostGIS] Indexing osm_barrierways on geohash took: 1m31.732333414s [May 15 06:03:28] [INFO] [PostGIS] Indexing osm_admin on geohash took: 2m48.561561564s [May 15 06:04:56] [INFO] [PostGIS] Indexing osm_waterareas on geohash took: 4m16.52188489s [May 15 06:05:31] [INFO] [PostGIS] Indexing osm_waterways on geohash took: 4m19.928695206s [May 15 06:05:33] [INFO] [PostGIS] Indexing osm_housenumbers on geohash took: 4m0.944478571s [May 15 06:05:33] [INFO] [PostGIS] Clustering on geometry took: 4m52.987107774s [May 15 06:05:33] pq: Geohash requires inputs in decimal degrees, got (-39 -90, -39 -90).

[torse]

Looks like an issue with coordinates on the south pole. Maybe this is related to #106?

[ImreSamu]

yes, probably related to the "South pole". because -this is the smallest data to replicate the error.

wget http://www.openstreetmap.org/api/0.6/node/436012592 -O spole.osm
osmium cat spole.osm -o spole.osm.pbf
....
+ /go/src/github.com/omniscale/imposm3/imposm3 import -mapping ../spole.yml -read ../spole.osm.pbf -srid 4326 -diff -write -optimize=true -overwritecache -deployproduction -connection postgis://localhost/imposm3dev
[May 19 01:46:46] [INFO] removing existing cache /tmp/imposm3
[May 19 01:46:46] [INFO] [     0] C:       0/s (1) N:       0/s (1) W:       0/s (0) R:      0/s (0)
[May 19 01:46:46] [INFO] Reading OSM data took: 329.953ms
[May 19 01:46:47] [INFO] [     0] C:       0/s ( 0.0%) N:       0/s (100.0%) W:       0/s (0) R:      0/s (0)
[May 19 01:46:47] [INFO] Writing OSM data took: 629.146ms
[May 19 01:46:47] [INFO] [PostGIS] Creating generalized tables took: 49µs
[May 19 01:46:47] [INFO] [PostGIS] Creating OSM id index on osm_spoint4326 took: 18.123ms
[May 19 01:46:47] [INFO] [PostGIS] Creating geometry index on osm_spoint4326 took: 5.269ms
[May 19 01:46:47] [INFO] [PostGIS] Creating geometry indices took: 23.78ms
[May 19 01:46:47] [INFO] Importing OSM data took: 653.152ms
[May 19 01:46:47] [INFO] [PostGIS] Indexing osm_spoint4326 on geohash took: 2.483ms
[May 19 01:46:47] [INFO] [PostGIS] Clustering on geometry took: 2.846ms
[May 19 01:46:47] pq: Geohash requires inputs in decimal degrees, got (-4.14066e-08 -90, -4.14066e-08 -90).

probably float conversion problem, because the imposm3 cache info of 'South Pole' "lat": -90.00000006261281 < -90

+ imposm3 query-cache -node 436012592
{
  "nodes": {
    "436012592": {
      "tags": {
        "name": "South Pole",
        "name:ast": "Polu Sur",
        "name:ba": "Көньяҡ ҡотоп",
        "name:de": "Südpol",
        "name:en": "South Pole",
        "name:es": "el Antártico",
        "name:fr": "pôle Sud",
        "name:it": "Polo antartico",
        "name:lt": "Pietų ašigalis",
        "name:pl": "Biegun południowy",
        "name:tt": "Көньяк котып",
        "name:uk": "Південний полюс",
        "name:zh": "南極點",
        "place": "locality",
        "wikipedia": "en:South Pole"
      },
      "lat": -90.00000006261281,
      "lon": -4.1406593709325534e-08
    }
  }
}

+ psql -d imposm3dev -c 'SELECT ST_AsText(geometry) , ST_IsValidDetail(geometry)   
  FROM osm_spoint4326  where st_y(geometry) < -90 ;'
                   st_astext                    | st_isvaliddetail 
------------------------------------------------+------------------
 POINT(-4.14065937093255e-08 -90.0000000626128) | (t,,)
(1 row)

I have created a (quick&dirty , minimal tested and not official ) patch, so if you can modify the /cache/binary/nodes.go code, you can test yourself.

diff --git a/cache/binary/nodes.go b/cache/binary/nodes.go
index e9c1a23..6f77f65 100644
--- a/cache/binary/nodes.go
+++ b/cache/binary/nodes.go
@@ -1,8 +1,12 @@
 package binary

+import (
+   "math"
+)
+
 func (this *Node) wgsCoord() (lon float64, lat float64) {
    lon = IntToCoord(this.GetLong())
-   lat = IntToCoord(this.GetLat())
+   lat = math.Max(IntToCoord(this.GetLat()), -90.0)
    return
 }

other workarounds :

• -srid 4326 -optimize=false
  • and fix with the coords with hand made SQL update
• prefilter the input data ( remove the 'south pole' , )
  • http://wiki.openstreetmap.org/wiki/Osmconvert#Clipping_based_on_Longitude_and_Latitude

[torse]

Ok, thanks for confirming and for the workaround. We simply adapted our import approach for now which works well:

1) Importing with -optimize=false 2) Running ST_IsValidDetail and updating geometry manually to -90 3) -optimize=true 4) -deploy-production

[JohnMorales]

I ran into this problem and can confirm that @ImreSamu's patch does work doing a full import -config config.json -read ./planet-latest.osm.pbf -write -diff -optimize works after manually applying the patch and rebuilding imposm3.

[olt]

Closing as the actual issue is #106