search

ppawel / openstreetmap-watch-list

OpenStreetMap Watch List (OWL) follows the minutely map updates and forms a base to build incremental analysis algorithms on.
http://wiki.openstreetmap.org/wiki/OWL_(OpenStreetMap_Watch_List)
BSD 3-Clause "New" or "Revised" License
25 stars 9 forks source link

Memory leak in the tiler #20

Open ppawel opened 11 years ago

ppawel commented 11 years ago

OOM killer kicked in again tonight... Some logs:

D, [2013-01-04T02:44:57.257553 #7989] DEBUG -- : W 152813734 (2)
D, [2013-01-04T02:44:57.280255 #7989] DEBUG -- :   tile_count = 2147504146
D, [2013-01-04T02:55:37.107015 #7989] DEBUG -- :   tiles_to_check = 1126144
D, [2013-01-04T03:01:23.219906 #7989] DEBUG -- :   Processing 2252464 tile(s)...

And then it's dead.

ppawel commented 11 years ago

I think it is not really this specific changeset (although it is huge) but some kind of memory leak in the tiler. When it runs on zark through the night you can clearly see memory used growing and growing until the OOM killer kills the tiler.

http://munin.openstreetmap.org/openstreetmap/zark.openstreetmap/memory.html

Not sure what this could be. GEOS objects seem to be automatically released. What else could it be? Need to investigate further.

Tried using heap_dump gem to check the heap contents but it requires Ruby newer than zark's 1.9.1...