omniscale / imposm3

Imposm imports OpenStreetMap data into PostGIS
http://imposm.org/docs/imposm3/latest/
Apache License 2.0
725 stars 159 forks source link

Schema "backup" already exists #84

Closed pnorman closed 8 years ago

pnorman commented 8 years ago

imposm issues CREATE SCHEMA "backup", but if a previous import was done this will fail, even if -removebackup has been used.

$ imposm3 import -mapping imposm3_mapping.json -connection='postgis://?prefix=NONE' -removebackup
[Jan 25 21:44:35] [INFO] Imposm took: 3.985717ms
$ psql -c '\dn backup'
  List of schemas
  Name  |  Owner
--------+----------
 backup | postgres
(1 row)
ImreSamu commented 8 years ago

Maybe this is related to a postgresql privilege or 'owner' problem with postgres superuser.

Sometimes I have some problem when I use imposm3 with different pgsql user. like ...

PGUSER=postgres PGPASSWORD=      imposm3 ...
PGUSER=osm      PGPASSWORD=osm   imposm3 ...

The first call create a new schema ( owner: postgres ) but the second call with 'osm' pguser can't see this via schemata view, so try to create: SQL Error: pq: schema "import" already exists in query CREATE SCHEMA "import"

As I know : imposm3 checking the schema existence via ' information_schema.schemata ' view with SELECT schema_name FROM information_schema.schemata and this give different result than \d ,

According to the information_schema.schemata doc "The view schemata contains all schemas in the current database that the _current user_ has access to (by way of being _the owner or having some privilege_)."

log of my simple test

+ PGUSER=postgres
+ PGPASSWORD=
+ import
+ /go/src/github.com/omniscale/imposm3/imposm3 import -mapping ./test/single_table_mapping.json -read ./parser/pbf/monaco-20150428.osm.pbf -write -overwritecache -connection postgis://localhost/imposm3dev
[Jan 26 13:33:00] [INFO] removing existing cache /tmp/imposm3
[Jan 26 13:33:01] [INFO] [reader] reading ./parser/pbf/monaco-20150428.osm.pbf with data till 2015-04-27 22:21:02 +0200 CEST
[Jan 26 13:33:01] [INFO] [     0] C:       0/s (17233) N:       0/s (931) W:       0/s (2398) R:      0/s (108)
[Jan 26 13:33:01] [INFO] Reading OSM data took: 437.587408ms
[Jan 26 13:33:01] [INFO] Writing OSM data took: 737.101131ms
[Jan 26 13:33:01] [INFO] [     0] C:       0/s ( 0.0%) N:       0/s (100.0%) W:       0/s (100.0%) R:      0/s (100.0%)
[Jan 26 13:33:01] [INFO] [PostGIS] Creating generalized tables took: 128.967µs
[Jan 26 13:33:01] [INFO] [PostGIS] Creating OSM id index on osm_all took: 50.459263ms
[Jan 26 13:33:01] [INFO] [PostGIS] Creating geometry index on osm_all took: 40.411767ms
[Jan 26 13:33:01] [INFO] [PostGIS] Creating geometry indices took: 91.195798ms
[Jan 26 13:33:01] [INFO] Importing OSM data took: 828.696977ms
[Jan 26 13:33:01] [INFO] Imposm took: 1.266429545s
+ echo ' ---------  osm pguser ------------- '
 ---------  osm pguser ------------- 
+ PGUSER=osm
+ PGPASSWORD=osm
+ import
+ /go/src/github.com/omniscale/imposm3/imposm3 import -mapping ./test/single_table_mapping.json -read ./parser/pbf/monaco-20150428.osm.pbf -write -overwritecache -connection postgis://localhost/imposm3dev
[Jan 26 13:33:01] [INFO] removing existing cache /tmp/imposm3
[Jan 26 13:33:02] [INFO] [reader] reading ./parser/pbf/monaco-20150428.osm.pbf with data till 2015-04-27 22:21:02 +0200 CEST
[Jan 26 13:33:02] [INFO] [     0] C:       0/s (17233) N:       0/s (931) W:       0/s (2398) R:      0/s (108)
[Jan 26 13:33:02] [INFO] Reading OSM data took: 397.523604ms
[Jan 26 13:33:02] SQL Error: pq: schema "import" already exists in query CREATE SCHEMA "import"
+ PGUSER=postgres Writing OSM data
+ PGPASSWORD=
+ dbtest
+ psql -c 'select current_user;'
 current_user 
--------------
 postgres
(1 row)

+ psql -c '\dn'
   List of schemas
   Name   |  Owner   
----------+----------
 import   | postgres
 public   | postgres
 topology | postgres
(3 rows)

+ psql -c 'select schema_name from information_schema.schemata ;'
    schema_name     
--------------------
 pg_toast
 pg_temp_1
 pg_toast_temp_1
 pg_catalog
 public
 information_schema
 topology
 import
(8 rows)

+ echo ' ---------  osm pguser ------------- '
 ---------  osm pguser ------------- 
+ PGUSER=osm
+ PGPASSWORD=osm
+ dbtest
+ psql -c 'select current_user;'
 current_user 
--------------
 osm
(1 row)

+ psql -c '\dn'
   List of schemas
   Name   |  Owner   
----------+----------
 import   | postgres
 public   | postgres
 topology | postgres
(3 rows)

+ psql -c 'select schema_name from information_schema.schemata ;'
    schema_name     
--------------------
 pg_catalog
 public
 information_schema
 topology
(4 rows)

the normal osm user can't see import scheme via information_schema.schemata

+ psql -c '\dn'
   List of schemas
   Name   |  Owner   
----------+----------
 import   | postgres
 public   | postgres
 topology | postgres
(3 rows)

+ psql -c 'select schema_name from information_schema.schemata ;'
    schema_name     
--------------------
 pg_catalog
 public
 information_schema
 topology
(4 rows)
ImreSamu commented 8 years ago

Probably need an extra schema check via pg_namespace and a correct error message to the user.

( see stackoverflow ... )

imposm3dev=> select nspname from pg_namespace ;
      nspname       
--------------------
 pg_toast
 pg_temp_1
 pg_toast_temp_1
 pg_catalog
 public
 information_schema
 topology
 import
(8 rows)
pnorman commented 8 years ago

Looks like my error was caused by not having permission to drop, and imposm3 swallowing the error so I didn't find that out.