search

tyrasd / osmtogeojson

convert osm to geojson
http://tyrasd.github.io/osmtogeojson/
MIT License
683 stars 112 forks source link

How to debug never stopping process #118

Open hyperknot opened 4 years ago

hyperknot commented 4 years ago

I am trying to use osmtogeojson to recreate an alternative to the recently shutdown Wambachers OSM Boundaries service.

I've exported boundary=administrative + admin_level=2..8 filtered osm files with osmium.

My problem is that while levels 2, 3, 4, 5 are processed withing a few minutes, level 6 is going on forever. On a 64 GB, 8 core server, it has been running for 66 hours now.

I think something is wrong or it went into an infinite loop somehow, but I cannot debug it.

Level 4 was 2.2 GB while Level 6 is 5.6 GB, but the difference shouldn't be that big. I mean Level 4 was processed in 4 minutes and for level 6 I'm waiting for 66 hours, so definitely something is not right.

If needed I can give the command which can replicate level 6 file. It's basically 2 command on a planet.pbf.

hyperknot commented 4 years ago

I couldn't debug what was causing this, but what I did was I've split the file to 10 segments and then they all completed correctly.

MrFoxPro commented 3 months ago

Same thing, processing admin_level=4 hang for an hour with max-old-space-size of 12228. That's why you don't write performance-sensetive programs in languages like javascript

upd: ran with Bun and after ~50min and 13Gb of RAM it finished