search

pgRouting / osm2pgrouting

Import tool for OpenStreetMap data to pgRouting database
https://pgrouting.org
GNU General Public License v2.0
288 stars 112 forks source link

std::bad_alloc while importing ... #222

Open postgresql007 opened 6 years ago

postgresql007 commented 6 years ago

hello ...

I have tried to import the German and Austrian map into PostgreSQL but it seems I got a memory related failure :( [hs@linuxpc osm]$ osm2pgrouting --f germany-latest.osm --dbname routing --username hs Execution starts at: Mon Mar 19 11:17:55 2018

       COMMAND LINE CONFIGURATION             *

Filename = germany-latest.osm Configuration file = /usr/share/osm2pgrouting/mapconfig.xml host = localhost port = 5432 dbname = routing username = hs password = schema= prefix = suffix = Don't drop tables Don't create indexes Don't add OSM nodes

Testing database connection: routing database connection successful: routing Connecting to the database connection success

Creating tables... TABLE: ways_vertices_pgr already exists. TABLE: ways_vertices_pgr already exists. TABLE: ways_vertices_pgr already exists. TABLE: ways_vertices_pgr already exists. Opening configuration file: /usr/share/osm2pgrouting/mapconfig.xml Parsing configuration

Exporting configuration ...

std::bad_alloc

I am running: [hs@linuxpc osm]$ osm2pgrouting -v This is osm2pgrouting Version 2.3.3

this is the standard package on Fedora with PostgreSQL 10. many thank for your work !

hans

dkastl commented 6 years ago

Germany is very big, and osm2pgrouting does stop working, when you run out of memory.

You could split everything into smaller data files and run osm2pgrouting with each of them and the --addnodes flag. Then it will append new nodes to the existing data.

It would be nice, if osm2pgrouting could handle this internally, but this needs some contribution or funding.

cvvergara commented 6 years ago

This issue shows a way around for getting the data into the database

https://github.com/pgRouting/osm2pgrouting/issues/201

postgresql007 commented 6 years ago

hi,

thank you very much for your superfast reply :). to me it looked like a bug. sorry for that. i added 300 GB of swap and it seems to work now. maybe just adding a different error message would already help a lot. i assume that other people had the same issue. maybe some …

“out of memory. Note: We will load all the data into RAM. consider adding more memory or swap space” … or something like that :).

many thanks,

    hans

On 19.03.2018, at 14:39, Daniel Kastl notifications@github.com wrote:

Germany is very big, and osm2pgrouting does stop working, when you run out of memory.

You could split everything into smaller data files and run osm2pgrouting with each of them and the --addnodes flag. Then it will append new nodes to the existing data.

It would be nice, if osm2pgrouting could handle this internally, but this needs some contribution or funding.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/pgRouting/osm2pgrouting/issues/222#issuecomment-374214446, or mute the thread https://github.com/notifications/unsubscribe-auth/AT3widQd_hrUpaM4Z5p_8wEnR5ar2hn9ks5tf7UIgaJpZM4Sv6O2.

-- Cybertec Schönig & Schönig GmbH Gröhrmühlgasse 26, A-2700 Wiener Neustadt Web: https://www.cybertec-postgresql.com https://www.cybertec-postgresql.com/

cayetanobv commented 6 years ago

Hi all,

In addition to everything said by @dkastl and @cvvergara , another tip that would improve your process @postgresql007 . You don't need a lot of metadata for PgRouting so if you follow this instructions you can greatly reduce OSM file size: https://github.com/pgRouting/osm2pgrouting/issues/221#issuecomment-372019082

Thanks!