search

navit-gps / navit

The open source (GPL v2) turn-by-turn navigation software for many OS
http://www.navit-project.org
Other
543 stars 173 forks source link

Maptool: Endless Loop on Phase 9 (generating multipolygons) #1128

Closed jkoan closed 2 years ago

jkoan commented 3 years ago

When Running maptool against the osm.pbf file attached below the Process keeps hanging and does only print the alarm info messages any 30secs

Expected Behavior

Maptool should continue and finish the Phase

Current Behavior

Runs in endless loop

Steps to Reproduce

greenland-latest.osm.pbf.gz Log from Gtihub Actions Log from my machine

  1. Download greenland-latest.osm.pbf.gz from above
  2. Gunzip the file
  3. Run the Command like normal: maptool --protobuf -i ~/Downloads/greenland-latest.osm.pbf ~/Downloads/greenland-latest.bin
  4. Wait untill "PROGRESS9: Processed 200 nodes (195 out) 0 ways 4185 relations 0 tiles" apperes multiple times

Context (Environment)

Github Actions: Standart ubuntu-latest runner PC: AMD Ryzen 2600 CPU; NVMe SSD; Manjaro Linux OS

Comment

Probably @metalstrolch could help here?

metalstrolch commented 3 years ago

Well... I need to deeper investigate what exactly happens, but on first view in both github actions and local run worker thread "4" does not finish making the main thread wait for ever. That never happened to me, but I'm mostly processing full planet. I'll try to debug if this is reproducible on my box.

metalstrolch commented 3 years ago

Can you retry with -T 1 in the meantime? This should limit the number of worker threads to 1. Shouldn't make much difference for Greenland as Greenland is quite smallish.

jkoan commented 3 years ago

-T 1 results in the same Behavior: maptool_phase_9loop-T_1.txt

I also debugged the Problem and it seems like its stuck in a loop within those lines: https://github.com/navit-gps/navit/blob/ea5a86beafcb52f54544e079ef7e4f8ac8b070f4/navit/maptool/osm.c#L2384-L2387

metalstrolch commented 3 years ago

Umm, I have the feeling maptool is doing right, but the OSM data got really complex recently.

It "stalls" at OSM relation ID 9488835 which was added recently to OSM. This one has 24960 member ways! It clearly outrules Aegyptian's lake nasser as one of the biggest OSM relation so far. So I assume to convert this it may take some time. Stay tuned if this turns out to be true.

metalstrolch commented 3 years ago

Ok, I was right. It finishes after around 2.5h on my machine (CoreI3, 3rd gen). So this is not a "bug". Be patient, and it finishes.

Fun fact is the resulting polygon of relation ID 9488835 doesn't even result in a viewable polygon in binfile, as its's tag do not result in any known type of Navit causing maptool to filter it out if -n is given. So converting it is quite useless.

It cannot be "solved" that processing huge multipolyons takes a lot of time. However I'll try to make "-n" parameter filter out multipolygons of "unknown" types before processing them so we at least do not process useless ones.

metalstrolch commented 3 years ago

See #1129 which would filter out relation 9488835 since it will only give "street_unknown" binfile type if you give -n on command line.

metalstrolch commented 2 years ago

Closing this as it was indeed no bug.