search

omniscale / imposm3

Imposm imports OpenStreetMap data into PostGIS
http://imposm.org/docs/imposm3/latest/
Apache License 2.0
723 stars 158 forks source link

Import custom relations #108

Closed edpop closed 8 years ago

edpop commented 8 years ago

Hello. I have my own non-geometry relations similar to associatedStreet.

<relation id="198" version = "1" changeset="28864" timestamp="2016-06-29T09:50:13Z">
  <member type="way" ref="123" role="road"/>
  <member type="way" ref="1234" role="road"/>
  <member type="way" ref="12345" role="building"/>
  <tag k="name" v="Arseniev street"/>
  <tag k="addr:type" v="street"/>
</relation>

I expect to get these tables:

street:
  id
  name

road:
  the_geom
  street_id

I try to get it reading an route example in documentation. My mapping.yml:

tables:
  road:
    type: relation_member
    mapping:
      addr:type: [street]
    columns:
    - name: the_geom
      type: geometry
    - name: street_id
      type: id

  street:
    type: relation
    mapping:
      addr:type: [street]
    columns:
    - name: osm_id
      type: id
    - name: name
      type: string
      key: name

After importing this tables are empty. Can you explain, please, what is wrong?

Ubuntu 14.04 Imposm3 0.2.0dev-20160615-d495ca4 PostgreSQL 9.5

ImreSamu commented 8 years ago

just a guess:

edpop commented 8 years ago

Yes, my relation is not assiciatedStreet, but similar (if some one converted it by imposm). The data is valid and JOSM reads it fine. This problem may be related to https://github.com/omniscale/imposm3/issues/12.

ImreSamu commented 8 years ago

my second guess :)

For me - this super relation mapping - is working for the real hungarian osm data ( tested now )

tags:
  load_all: true
  exclude:
  - created_by
  - source

tables:
  x_all_relation_members:
    type: relation_member
    fields:
    - name: osm_id
      type: id
    - name: member
      type: member_id
    - name: index
      type: member_index
    - name: role
      type: member_role
    - name: type
      type: member_type
    - name: geometry
      type: geometry
    - name: relname
      key: name
      type: string
    - name: name
      key: name
      type: string
      from_member: true
    mapping:
      __any__:
      - __any__

  x_all_relations:
    type: relation
    fields:
    - name: osm_id
      type: id
    - key: type
      name: type
      type: string
    - key: name
      name: name
      type: string      
    - name: tags
      type: hstore_tags
    mapping:
      __any__:
      - __any__

and the assiciatedStreet is in the list : 


imposm3dev=> SELECT type , count(*) as count 
imposm3dev-> FROM osm_x_all_relations
imposm3dev-> GROUP BY type
imposm3dev-> ORDER BY count desc ;
       type        | count 
-------------------+-------
 multipolygon      | 34151
 route             |  6501
 restriction       |  6037
 boundary          |  3951
 public_transport  |  1455
 route_master      |   873
 turnlanes:turns   |   357
 associatedStreet  |   156
 turnlanes:lengths |    81
 street            |    41
 waterway          |    33
 site              |    27
 destination_sign  |    18
 network           |    15
                   |    14
 building          |     9
 superroute        |     9
 link              |     8
 multilinestring   |     3
 bridge            |     2
 enforcement       |     2
 network_of_routes |     2
 abandoned:route   |     2
 level             |     1
 label             |     1
 person            |     1
 line              |     1
 classification    |     1
 microwave         |     1
 circuit           |     1
(30 rows)

and the same ~ numbers here : http://taginfo.openstreetmap.hu/relations

The osm_x_all_relation_members table - TOP10 role : 

imposm3dev=> 
imposm3dev=> SELECT role, count(*) as count 
imposm3dev-> FROM osm_x_all_relation_members
imposm3dev-> GROUP BY role
imposm3dev-> ORDER BY count desc
imposm3dev-> LIMIT 10 ;
     role     | count  
--------------+--------
              | 294759
 outer        | 205602
 stop         |  50170
 inner        |  15929
 from         |   6908
 to           |   6831
 via          |   6390
 platform     |   4932
 admin_centre |   3813
 house        |   1762
(10 rows)
ImreSamu commented 8 years ago

the load_all: true is a workaround

edpop commented 8 years ago

Thanks, that working! I got street relations as I wanted.

But now I can't import cities, they are relations with type: multipolygon consists of many inner and outer ways.

My address hierarchy: street (non-geometry relation) → city (multipolygon) → region_district (multipolygon) → region (multipolygon).

All this objects, except streets, are valid multipolygons in OSM specification like buildings, just larger.

osm2pgsql converts all of this to multipolygons correctly (Is this issue solved?).

Tried to import as relation and as polygon - empty table.

Option 1:

tags:
  load_all: true

tables:
  city:
    type: polygon
    mapping:
      addr:type:
      - city
    columns:
    - name: tags
      type: hstore_tags
    - name: the_geom
      type: geometry

Option 2:

tags:
  load_all: true

tables:
  city:
    type: relation
    mapping:
      addr:type:
      - city
    columns:
    - name: tags
      type: hstore_tags

When I import all relations (type: relation, mapping: any: any), there are only streets and some multipolygon buildings.

ImreSamu commented 8 years ago

But now I can't import cities, they are relations with type: multipolygon consists of many inner and outer ways.

For me , the example admin mapping (example-mapping.yml) is working perfectly for all Hungarian OSM admin data ( country border, regions, cities ) 

  admin:
    fields:
    - name: osm_id
      type: id
    - name: geometry
      type: geometry
    - key: name
      name: name
      type: string
    - name: type
      type: mapping_value
    - key: admin_level
      name: admin_level
      type: integer
    mapping:
      boundary:
      - administrative
    type: polygon

All this objects, except streets, are valid multipolygons in OSM specification like buildings, just larger.

As I know the type: polygon mapping created from :

edpop commented 8 years ago

The problem was related with nodes order in osm file. Ordered by id and it works.