Converts OSM planet.osm data to a PostgreSQL / PostGIS database suitable for specific applications like rendering into map tiles by Mapnik or geocoding with Nominatim.
osm2pgsql currently supports two different database schemas 1) A database schema that is optimized for ease of rendering by Mapnik. 2) A database schema that is optimized for geocoding with Nominatim, emphasizing the spatially hierarchical organizations of objects.
Both schemas were specifically optimized for the purpose they were intended for and they may therefore be less suitable for other general purpose processing. Nevertheless, the rendering schema might be useful for other purposes as well, and has been used for a variety of additionally purposes.
For a broader view of the whole map rendering tool chain see http://wiki.openstreetmap.org/index.php/Mapnik http://wiki.openstreetmap.org/index.php/Osm2pgsql http://wiki.openstreetmap.org/index.php/Slippy_Map
You may find that the wiki pages are more up to date than this readme and may include answers to issues not mentioned here.
Any questions should be directed at the osm dev list http://wiki.openstreetmap.org/index.php/Mailing_lists
The latest source code is available in the OSM git repository on github and can be downloaded as follows:
$ git clone git://github.com/openstreetmap/osm2pgsql.git
The code is written in C and C++ and relies on the libraries below:
To make use of the database generated by this tool you will probably also want to install:
Make sure you have installed the development packages for the libraries mentioned in the requirements section and a C and C++ compiler.
e.g. on Fedora:
on Debian:
To use lua tag transforms you need a lua interpreter and development packages installed on your system. These are normally lua5.2 and liblua5.2-dev
On most Unix-like systems the program can be compiled by running './autogen.sh && ./configure && make'.
You must create a PostgreSQL user and a database with the PostGIS functions enabled. This requires access as the database administrator, normally the 'postgres' user.
PostgreSQL 9.1 and PostGIS 2.0 or later are strongly suggested for databases in production. It is generally best to run the latest released versions if possible. PostgreSQL 8.4 and PostGIS 1.5 will work but are substantially slower. Additionally, PostGIS 2.0 contains enhancements that increase reliability as well as add new features that style sheet authors can use.
The default name for this database is 'gis' but this may be changed by using the osm2pgsql --database option.
If the
Some example commands are given below but you may find this wiki page has more up to data information: http://wiki.openstreetmap.org/wiki/Mapnik/PostGIS
$ sudo -u postgres createuser
Adding the PostGIS extensions.
$ sudo -u postgres psql -d
On older versions of PostGIS you will have to run .sql scripts. Note the location of the files may vary.
$ sudo -u postgres psql -d
Next we need to give the
$ sudo -u postgres psql -d
The 900913 is not normally included with PostGIS. To add it you should run:
$ sudo psql -u postgres psql -d
If you want to use hstore support then you will also need to enable the PostgreSQL hstore-new extension.
$ sudo -u postgres psql -d
On PostgreSQL versions before 9.1 you need to install the hstore-new extension instead
$ sudo -u postgres psql -d
Now you can run osm2pgsql to import the OSM data. This will perform the following actions:
1) osm2pgsql connects to database and creates the following 4 tables when used with the default output back-end (pgsql):
If you are using --slim mode, it will create the following additional 3 tables:
2) Runs a parser on the input file (typically planet-latest.osm.pbf) and processes the nodes, ways and relations.
3) If a node has a tag declared in the style file then it is added to planet_osm_point. If it has no such tag then the position is noted, but not added to the database.
4) Ways are read in converted into WKT geometries by using the positions of the nodes read in earlier. If the tags on the way are listed in the style file then the way will be written into the line or roads tables.
5) If the way has one or more tags marked as 'polygon' and forms a closed ring then it will be added to the planet_osm_polygon table.
6) The relations are parsed. Osm2pgsql has special handling for a limited number of types: multipolygon, route, boundary The code will build the appropriate geometries by referencing the members and outputting these into the database.
7) Indexes are added to speed up the queries by Mapnik.
For an efficient operation of PostgreSQL you will need to tune the config parameters of PostgreSQL from its default values. These are set in the config file at /etc/postgresql/9.1/main/postgresql.conf
The values you need to set will depend on the hardware you have available, but you will likely need to increase the values for the following parameters:
Depending on the command-line switches you can select which projection you want the database in. You have three choices:
4326: The standard lat/long coordinates 900913: The spherical Mercator projection, used by TileCache, Google Earth etc. 3395: The legacy (broken) WGS84 Mercator projection
Depending on what you're using one or the other is appropriate. The default Mapnik style (osm.xml) assumes that the data is stored in 900913 and this is the default for osm2pgsql.
Combining the -v and -h switches will tell about the exact definitions of the projections.
In case you want to use some completely different projection there is the -E
option. It will initialize the projection as +init=epsg:
If you wish to access the data from the database then the queries below should give you some hints. Note that these examples all use the 'latlong' projection which is not the default.
$ psql gis gis=> \d List of relations Schema | Name | Type | Owner --------+--------------------+-------+---------- ... public | planet_osm_line | table | jburgess public | planet_osm_point | table | jburgess public | planet_osm_polygon | table | jburgess public | planet_osm_roads | table | jburgess ...
gis=> \d planet_osm_line Table "public.planet_osm_line" Column | Type | Modifiers -----------+----------+----------- osm_id | integer | name | text | place | text | landuse | text | ... [ lots of stuff deleted ] ... way | geometry | not null z_order | integer | default 0
Each of the tables contains a subset of the planet.osm file representing a particular geometry type
Point contains nodes which have interesting tags e.g. place=city, name=London
Line contains ways with interesting tags e.g. highway=motorway, ref=M25
Polygon contains ways which form an enclosed area e.g. landuse=reservoir
The DB columns are used as follows:
Querying specific data requires knowlege of SQL and the OSM key/value system, e.g.
gis=> select osm_id,astext(way),name from planet_osm_point where amenity='cinema' limit 5; osm_id | astext | name ----------+-------------------------------------------+-------------------- 26236284 | POINT(-79.7160836579093 43.6802306464618) | 26206699 | POINT(51.4051989797638 35.7066045032235) | Cinema Felestin 26206700 | POINT(51.3994885141459 35.7058460359352) | Cinema Asr-e Jadid 20979630 | POINT(151.225781789807 -33.8943079539886) | Paris Cinema 20979684 | POINT(151.226855394904 -33.8946830511095) | Hoyts (5 rows)
Mapnik renders the data in each table by applying the rules in the osm.xml file.
How could I get e.g. all highways in a given bounding box?
The 'way' column contains the geo info and is the one which you need to use in your WHERE clause. e.g.
gis=> select osm_id,highway,name from planet_osm_line where highway is not null and way && GeomFromText('POLYGON((0 52, 0.1 52, 0.1 52.1, 0 52.1, 0 52))',4326);
osm_id | highway | name ---------+--------------+------------------ 4273848 | unclassified | 3977133 | trunk | to Royston (tbc) 4004841 | trunk | 4019198 | trunk | 4019199 | trunk | 4238966 | unclassified |
See the Postgis docs for details, e.g. http://postgis.net/docs/manual-2.0/reference.html