Framstag / libosmscout

Libosmscout is a C++ library for offline map rendering, routing and location lookup based on OpenStreetMap data
Other
260 stars 81 forks source link

Import routing data only #915

Closed zedxxx closed 4 years ago

zedxxx commented 4 years ago

I want to use only routing functionality in my application. Using Import tool I can convert small *.pbf files fast enough, but with large files (whole Europe, for example) it take ages. So my question is: Is it possible to skip unnecessary steps (if any) in Import process (one of this 27-28 steps) to speed up data importing?

Karry commented 4 years ago

Hi.

You can remove some data types from map.ost to make import faster. You may removes almost all types that are marked with keyword PATH (some other types are required by code import code itself, but I don't know exact list).

Another way howto speedup import is running just selected steps. On this diagram https://osmscout.karry.cz/europe/czech-republic-21-20200922-1128/stats.svg you may track which temporary files and steps are required for "RouteDataGenerator". But I think that you cannot skip something big...

zedxxx commented 4 years ago

Can you explain what is it map.ost and how can I remove something from it? I use *.osm.pbf from here http://download.geofabrik.de/europe.html

Karry commented 4 years ago

During import, you have to provide --typefile <somepath>/map.ost argument to Import tool. That map.ost file is placed in stylesheets/map.ost in repository. It is text file with definition of types that are extracted from OSM data file and used in database. When you are interested just for router feature, you don't need to import areas, nodes and even ways that are not used later for generating router data files...

Syntax of that file is described here: http://libosmscout.sourceforge.net/documentation/typedef/

Framstag commented 4 years ago

If you want to leave out steps, you can use -s or -e to limit the number of steps run. Note though, that for a correct database most steps have a dependency on earlier steps. This is the reason, why it is safer to keep all steps but reduce the list of imported data,

Note also that it is better, to set individual types to "IGNORE" instead of removing it. Since some objects could map to multiple types, removing some may result in wrong mapping to other types. IGNORE removes matching objects without importing them, prohibiting later matches,

zedxxx commented 4 years ago

Ok, I got it. I will try to edit the map.ost by adding IGNORE and report the results.

zedxxx commented 4 years ago

My stylesheet optimized for routing only: route.ost

Import is ~3x faster with this file.

zedxxx commented 4 years ago

Also, steps 18-21/22 can be safely skipped. If you need street names for your route, than don't skip step 22 (location.idx generation).

Framstag commented 4 years ago

I'm glad to hear things are working out for you. Note that there a number of optimizations options for import and at runtime. If you have further questions, just ask...!