Routing issue with generated mwm files

[khalilst]

After creating binaries for OSRM and Generator_Tool, I have created some mwm files for couple of Iran provinces. Then copied them into MapsWithMe directory in Android and maps displayed like a charm. But there were 2 problems:

1. MapsMe in its startup, says you need download Iran_North.mwm (size 54MB) - I can skip this problem by creating null file in same directory with exact name

2. Routing is not working and it requests for downloading Iran_North again. But the generated map completely inside in Iran_north and it doesn't need it.

Did I missed something? I've used this command: mim/tools/unix/generate_mwm.sh source.pbf any_random_string_asdf

And compiled it using this command: CONFIG=gtool omim/tools/unix/build_omim.sh -cro

[Zverik]

Alas routing only works when your map is named like the app suggests. Try renaming it to Iran_North.mwm.

[khalilst]

Thank you @Zverik , Bike and foot routing works with your tip, But selecting car routing raise an error: Unable to locate route Unable to create route. Please adjust your starting point or your destination. Selecting different routing after above error, works fine. What's wrong with that?

And is there any option to ignore this method? I mean forcing download files like Iran_North. I want to provide daily update for Iran Provinces, but some people like using multiple provinces and your tip is not working for them.

Is there any setting to ignore using official maps?

[21212124]

Hello, I am missing source.pbf, where is it? How did you do the routing? Can you explain me? thank you

[Locke]

Hello,

the parameter for "any_random_string_asdf" has been removed in b79686c73. However I still can't generate intra-mwm routing without generating inter-mwm routing; @Zverik mentioned to me this is a known bug.

@khalilst You should try out that approach (using the BORDERS_PATH variable).

[khalilst]

@Locke I tried some tools like osm2poly.pl to generate poly file from osm file. But they throw an exception. I know this is kinda off-topic, but I appreciate to find a working tool.

And why BORDERS_PATH is called before the command, I'm not familiar passing param this way?!

[Locke]

A few general notes and steps about pbf and poly files, and the routing generation in general:

• the pbf file contains the exported osm map data (like roads, buildings, POIs, ...)
• the poly file describes the area of the pbf file (i.e. a polygon around it)

One possible way to create a poly file is to draw an area in an editor for osm. In that case you'll have a file in osm format containing only that single way and you can use osm2poly.pl to convert this single way from the osm format to the poly format. It is not possible to automatically create a poly file from the exported osm data, as the exported osm data will have ways, esp. roads, reaching out of the original exported area. @khalilst

Another way of getting a poly file is to just use a suiting one from https://github.com/mapsme/omim/tree/master/data/borders

The pbf file is then created from the the polygon as filter around some osm data (mostly the raw osm planet file with possibly some live updates. (the creation is too much for here describe. If you know what I'm talking about you probably can figure this out on your own or already know how this works, if not just see the next alternative and don't bother about creating your own poly and pbf files)

@21212124 If you don't want to bother with creating your own poly and pbf files you can download existing ones. In my case I used https://download.geofabrik.de/north-america/us/puerto-rico.html

Now that we have both the data and the area we can use them to convert them to a mwm file for maps.me. Just converting the map data doesn't need the polygon around it, but it is needed to create the routing index. If the routing index is not created the app will crash if you try to calculate a route @arcticbarra

// from here on you need a unix-like environment, for example gnu/linux. I'm using debian as an operating system, but it might be possible to use os x or the linux subsystem for windows as well. I haven't tried those..

The conversion from pbf to mwm is done using the generate_mwm.sh script with the pbf file path as command line argument. Since b79686c the parameter for "any_random_string_asdf" has been removed and the routing index for routing within the wmw should be created automatically, see also #7215.

However there is a known bug in the routing generation, due to it the routing index is not stored.

The workaround is to create the routing index also for inter-mwm routing. Like I said earlier the pbf file contains roads reaching out of the polygon - in order to calculate routes between multiple mwm files such roads need to be "connected".

The generate_mwm.sh script heavily depends on environment variables for additional arguments. To enable inter-mwm routing the environment variable BORDERS_PATH has to be set to a folder that contains at least the poly file for your pbf file, and optionally additional poly files for surrounding / additional pbfs. Note: that folder must not be named borders, I just use myborders. Note: the poly files must have the same name as the pbf files. @khalilst the bash syntax is myenvarg=myenvdata mycommand to supply a environment variable named myenvarg to the command mycommand. It is not a command line argument.

[enriquebrgn]

Thank you for your explanation, @Locke.

I just tried running BORDERS_PATH=myborders tools/unix/generate_mwm.sh myborders/Mexico_Central_East.osm.pbf and it still crashes when trying to generate a route. In the myborders folder I have my Mexico_Central_East.osm.pbf file and my Mexico_Central_East.poly file, both downloaded from https://download.geofabrik.de/north-america/mexico.html. From the output it seems it is indeed generating an index. LOG TID(1) INFO 16.2016 generator/routing_index_generator.cpp:369 BuildRoutingIndex() Routing section created: 6006375 bytes, 1295749 roads, 2310226 joints, 4816170 points

Any ideas on what is going wrong?

Thank you for your help.

[Zverik]

BORDERS_PATH=data/borders tools/unix/generate_mwm.sh Mexico_Central_East.osm.pbf You need all the borders.

[khalilst]

@Locke Thanks for your great guide I created poly files using this website for a province. But car routing resulted to application error. Here is the error image.

@Zverik Is it possible to pull request to add poly files of Iran provinces to this folder. I'm volunteer to daily generate mwm files in my own server and provide it to public for these provinces or whole country, of course after resolving this issue ;) Actually it encourages some mappers to see their works in MapsWithMe and totally it has a positive result in popularity of this software.

[enriquebrgn]

Hi @Zverik, I spent the last 24 hours generating the routing index with the whole borders folder but sadly, it still crashes when trying to generate a route. Any other ideas on how it might work?

Thanks in advance!

[Zverik]

You could have spent an entire week and still I won't be able to help you without the exact command line and script alterations you made.

[enriquebrgn]

I ran BORDERS_PATH=data/borders tools/unix/generate_mwm.sh Mexico_Central_East.osm.pbf The only thing that changed with my previous command was that now I passed the whole borders instead of just the Mexico one

[Locke]

I think the last log line should be Cross mwm section generated and not only Routing section created.

Also you might want to try out the "release-75" branch if you are using the 7.5 app. Today I compiled the new "release-76" branch and the generated mwm crashed the 7.5.4 app on routing (I didn't try out a 7.6 beta app, so this might be expected).

[Locke]

As the 7.6 android app is released I tried it out with the generator tool compiled with both the release-75 and release-76 branch. In both cases the routing didn't crash, so I don't see any regression.

@arcticbarra @khalilst can you update your apps and make sure you are using the generator tool compiled from the corresponding branch?

[khalilst]

@Locke I updated it from google play, still v7.5.4 and it didn't work. Can you share plz the link for latest version

[adrianalin]

Hello all, i also have difficulties in generating the mwm (for Romania). I am trying with the desktop app (branch release-76). Here are the steps i took:

1. I downloaded the romania-latest.osm.pbf from http://download.geofabrik.de/europe/romania.html and renamed it to Romania.pbf.

2. Downloaded the poly file from geofabrik, and renamed it to Romania.poly, and saved it in omim/data/borders.

3. run omim/tools/unix/generate_mwm.sh Romania.pbf.

4. Copy the generated Romania.mwm (and renamed it to Romania_North_West.mwm) to omim/data/170817

5. Run the desktop app, the map shows nicely, but when i set a start point and an end point on the map (in order to calculate route) the app crashes :

Debugging starts LOG TID(1) DEBUG 3.9937e-05 platform/platform_linux.cpp:192 Platform() Resources directory: /home/adrian/Downloads/original_omim/build-omim-Desktop_Qt_5_8_0_GCC_64bit-Debug/out/debug/../../../omim/data/ LOG TID(1) DEBUG 9.7498e-05 platform/platform_linux.cpp:193 Platform() Writable directory: /home/adrian/Downloads/original_omim/build-omim-Desktop_Qt_5_8_0_GCC_64bit-Debug/out/debug/../../../omim/data/ LOG TID(1) DEBUG 0.000105038 platform/platform_linux.cpp:194 Platform() Tmp directory: /tmp/ LOG TID(1) DEBUG 0.000110241 platform/platform_linux.cpp:195 Platform() Settings directory: /home/adrian/.config/MapsWithMe/ LOG TID(1) DEBUG 0.000175616 platform/platform_linux.cpp:196 Platform() Client ID: 88267a4574c5490bbccf1c2e0094c6c4 14/nov/2017:12:30:27 Alohalytics: Enabled debug mode. LOG TID(1) INFO 1.01656 platform/string_storage_base.cpp:26 StringStorageBase() Settings path: /home/adrian/.config/MapsWithMe/settings.ini LOG TID(1) INFO 1.16177 Loaded countries list for version: 171020 LOG TID(2) INFO 1.16634 tracking/reporter.cpp:81 Run() Tracking Reporter started LOG TID(1) INFO 1.1743 editor/user_stats.cpp:76 UserStatsLoader() There is no cached user stats info in settings LOG TID(1) DEBUG 1.17439 indexer/classificator_loader.cpp:45 Load() Reading of classificator started LOG TID(1) DEBUG 1.39414 indexer/classificator_loader.cpp:67 Load() Reading of classificator finished LOG TID(1) DEBUG 1.39418 map/framework.cpp:443 Framework() Classificator initialized LOG TID(1) DEBUG 1.78057 map/framework.cpp:449 Framework() Country info getter initialized LOG TID(1) DEBUG 1.78125 map/framework.cpp:452 Framework() UGC initialized LOG TID(3) WARNING 1.7814 ugc/storage.cpp:202 Load() Exception while reading file: /home/adrian/.config/MapsWithMe/index.json reason: Reader::OpenException /home/adrian/Downloads/original_omim/omim/coding/internal/file_data.cpp:59, "/home/adrian/.config/MapsWithMe/index.json; Read; No such file or directory" LOG TID(1) INFO 1.81106 search/processor.cpp:221 SetPreferredLocale() New preferred locale: en LOG TID(1) DEBUG 1.81275 search/processor.cpp:238 SetInputLocale() New input locale: en LOG TID(1) INFO 1.81281 search/processor.cpp:221 SetPreferredLocale() New preferred locale: en_US LOG TID(1) DEBUG 1.81282 search/processor.cpp:238 SetInputLocale() New input locale: en_US LOG TID(1) DEBUG 1.82029 map/framework.cpp:455 Framework() Search engine initialized LOG TID(1) INFO 1.86208 storage/storage.cpp:276 RegisterAllLocalMaps() Found file: Romania_North_West in directory: /home/adrian/Downloads/original_omim/build-omim-Desktop_Qt_5_8_0_GCC_64bit-Debug/out/debug/../../../omim/data/170817 LOG TID(1) INFO 1.86214 storage/storage.cpp:276 RegisterAllLocalMaps() Found file: World in directory: LOG TID(1) INFO 1.86216 storage/storage.cpp:276 RegisterAllLocalMaps() Found file: WorldCoasts in directory: LOG TID(1) INFO 1.86217 storage/storage.cpp:276 RegisterAllLocalMaps() Found file: WorldCoasts_obsolete in directory: /home/adrian/Downloads/original_omim/build-omim-Desktop_Qt_5_8_0_GCC_64bit-Debug/out/debug/../../../omim/data/ LOG TID(1) INFO 1.86219 storage/storage.cpp:276 RegisterAllLocalMaps() Found file: minsk-pass in directory: /home/adrian/Downloads/original_omim/build-omim-Desktop_Qt_5_8_0_GCC_64bit-Debug/out/debug/../../../omim/data/ LOG TID(1) INFO 1.904 map/framework.cpp:211 RegisterMap() Loading map: Romania_North_West LOG TID(1) INFO 1.90439 map/framework.cpp:211 RegisterMap() Loading map: World LOG TID(1) INFO 1.9046 map/framework.cpp:211 RegisterMap() Loading map: WorldCoasts LOG TID(1) INFO 1.90474 map/framework.cpp:211 RegisterMap() Loading map: minsk-pass LOG TID(1) DEBUG 1.9049 map/framework.cpp:463 Framework() Maps initialized LOG TID(1) DEBUG 1.90494 map/framework.cpp:470 Framework() Storage initialized LOG TID(4) INFO 1.90582 diff_scheme/diff_scheme_checker.cpp:132 Check() Request to diffs server failed. Code = -1, redirection = 0 LOG TID(1) DEBUG 1.92854 map/framework.cpp:483 Framework() Routing engine initialized LOG TID(1) INFO 1.92859 map/framework.cpp:485 Framework() System languages: en_US LOG TID(1) INFO 1.92864 indexer/osm_editor.cpp:276 LoadMapEdits() Loaded 0 modified, 0 created, 0 deleted and 0 obsolete features. LOG TID(1) INFO 1.92865 map/framework.cpp:495 Framework() Editor initialized LOG TID(1) DEBUG 1.92922 map/framework.cpp:502 Framework() Transliterators initialized libpng warning: iCCP: known incorrect sRGB profile libpng warning: iCCP: known incorrect sRGB profile libpng warning: iCCP: known incorrect sRGB profile LOG TID(1) WARNING 2.64902 map/traffic_manager.cpp:97 SetEnabled() Invalid attempt to disable traffic manager, it's already disabled , doing nothing. LOG TID(5) INFO 2.65039 drape_frontend/frontend_renderer.cpp:1802 Do() Start routine. LOG TID(5) INFO 2.65042 drape_frontend/frontend_renderer.cpp:1748 OnContextCreate() On context create. LOG TID(5) INFO 2.65378 drape/support_manager.cpp:22 Init() Renderer = Mesa DRI Intel(R) HD Graphics 520 (Skylake GT2) Api = OpenGLES2 Driver version = 3.0 Mesa 12.0.6 14/nov/2017:12:30:28 Alohalytics: LogEvent: GPU = Mesa DRI Intel(R) HD Graphics 520 (Skylake GT2) LOG TID(5) INFO 2.65393 drape/support_manager.cpp:55 Init() Max line width = 7 LOG TID(5) INFO 2.65399 drape/gpu_program_manager.cpp:28 Init() VTF enabled LOG TID(6) INFO 2.68565 drape_frontend/backend_renderer.cpp:546 Do() Start routine. LOG TID(6) INFO 2.68569 drape_frontend/backend_renderer.cpp:517 OnContextCreate() On context create. LOG TID(6) INFO 2.69008 platform/platform_unix_impl.cpp:116 GetSystemFontNames() Found usable system font /usr/share/fonts/truetype/padauk/Padauk.ttf with file size 472488 LOG TID(6) INFO 2.69015 platform/platform_unix_impl.cpp:116 GetSystemFontNames() Found usable system font /usr/share/fonts/truetype/ttf-khmeros-core/KhmerOS.ttf with file size 265552 LOG TID(6) INFO 2.69019 platform/platform_unix_impl.cpp:116 GetSystemFontNames() Found usable system font /usr/share/fonts/truetype/tlwg/Umpush.ttf with file size 83404 LOG TID(6) INFO 2.69039 platform/platform_unix_impl.cpp:116 GetSystemFontNames() Found usable system font /usr/share/fonts/truetype/freefont/FreeSans.ttf with file size 1563256 LOG TID(6) INFO 2.69047 platform/platform_unix_impl.cpp:116 GetSystemFontNames() Found usable system font /usr/share/fonts/truetype/abyssinica/AbyssinicaSIL-R.ttf with file size 602664 LOG TID(6) INFO 2.69049 platform/platform.cpp:164 GetFontNames() Available font files: [12: 01_dejavusans.ttf 02_droidsans-fallback.ttf 03_jomolhari-id-a3d.ttf 04_padauk.ttf 05_khmeros.ttf 06_code2000.ttf 07_roboto_medium.ttf /usr/share/fonts/truetype/padauk/Padauk.ttf /usr/share/fonts/truetype/ttf-khmeros-core/KhmerOS.ttf /usr/share/fonts/truetype/tlwg/Umpush.ttf /usr/share/fonts/truetype/freefont/FreeSans.ttf /usr/share/fonts/truetype/abyssinica/AbyssinicaSIL-R.ttf ] LOG TID(6) DEBUG 2.79232 drape/texture_manager.cpp:473 Init() Patterns texture size = 512 64 LOG TID(6) DEBUG 2.79265 drape/texture_manager.cpp:487 Init() Colors texture size = 64 64 LOG TID(7) DEBUG 13.4446 drape_frontend/metaline_manager.cpp:83 OnTaskFinished() Metalines prepared: MwmId [Romania_North_West] LOG TID(8) INFO 85.4041 routing/async_router.cpp:302 CalculateRoute() Calculating the route. checkpoints: Checkpoints(46.777482 23.610482, 46.776220 23.611969, passed: 0) startDirection: m2::Point<d>(0, 0) router name: astar-bidirectional-Car LOG TID(8) INFO 96.1832 routing/index_graph_loader.cpp:85 Load() routing section for Romania_North_West loaded in 3.9104 seconds ASSERT FAILED routing/cross_mwm_osrm_graph.hpp:72 CHECK(mapping->IsValid()) Mwm: CountryFile [Romania_North_West] was not loaded. MAPS.ME: /home/adrian/Downloads/original_omim/omim/routing/cross_mwm_osrm_graph.hpp:72: void routing::CrossMwmOsrmGraph::LoadWith(routing::NumMwmId, Fn&&) [with Fn = const routing::CrossMwmOsrmGraph::LoadSegmentMaps(routing::NumMwmId)::<lambda(const TRoutingMappingPtr&)>&; routing::NumMwmId = short unsigned int]: Assertionfalse' failed. `

Any idea what am i doing wrong?