search

MapServer / mapcache

MapCache source code directory. Please submit pull requests to the 'main' branch.
https://mapserver.org/mapcache
Other
137 stars 96 forks source link

mapcache_seed leaks projectionObj data #97

Open dmorissette opened 10 years ago

dmorissette commented 10 years ago

Using mapcache_seed / MapCache 1.2.1 we have noticed what seems like a leak of projectionObj data, caused by the call to msLoadProjectionString() in _mapcache_source_mapserver_render_map() (source_mapserver.c)

We will try to produce a self-contained testcase, but in the meantime, here is what Valgrind reports for a short test and a longer run on the same maps:

Short test:

==566== 10,065 (4,920 direct, 5,145 indirect) bytes in 15 blocks are definitely lost in loss record 72 of 78
==566==    at 0x4C2ABED: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==566==    by 0xA592AB8: pj_malloc (in /usr/lib64/libproj.so.0.7.0)
==566==    by 0xA579461: pj_merc (in /usr/lib64/libproj.so.0.7.0)
==566==    by 0xA591D74: pj_init_ctx (in /usr/lib64/libproj.so.0.7.0)
==566==    by 0x7242801: msProcessProjection (in /usr/lib64/libmapserver-6.2.0.so)
==566==    by 0x7242970: msLoadProjectionString (in /usr/lib64/libmapserver-6.2.0.so)
==566==    by 0x4E62E67: _mapcache_source_mapserver_render_map (in /opt/mapcache-1.2.1-ms-6.2-dev/lib/libmapcache.so.1.2.1)
==566==    by 0x4E65860: mapcache_tileset_render_metatile (in /opt/mapcache-1.2.1-ms-6.2-dev/lib/libmapcache.so.1.2.1)
==566==    by 0x4048C8: seed_worker (in /opt/mapcache-1.2.1-ms-6.2-dev/bin/mapcache_seed)
==566==    by 0x404A59: seed_thread (in /opt/mapcache-1.2.1-ms-6.2-dev/bin/mapcache_seed)
==566==    by 0xB9A7E0D: start_thread (in /lib64/libpthread-2.15.so)
==566==    by 0x7B692CC: clone (in /lib64/libc-2.15.so)
==566== 

==566== 495 (240 direct, 255 indirect) bytes in 15 blocks are definitely lost in loss record 50 of 78
==566==    at 0x4C2ABED: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==566==    by 0x725421D: msSmallMalloc (in /usr/lib64/libmapserver-6.2.0.so)
==566==    by 0x7242BBF: msLoadProjectionString (in /usr/lib64/libmapserver-6.2.0.so)
==566==    by 0x4E62E67: _mapcache_source_mapserver_render_map (in /opt/mapcache-1.2.1-ms-6.2-dev/lib/libmapcache.so.1.2.1)
==566==    by 0x4E65860: mapcache_tileset_render_metatile (in /opt/mapcache-1.2.1-ms-6.2-dev/lib/libmapcache.so.1.2.1)
==566==    by 0x4048C8: seed_worker (in /opt/mapcache-1.2.1-ms-6.2-dev/bin/mapcache_seed)
==566==    by 0x404A59: seed_thread (in /opt/mapcache-1.2.1-ms-6.2-dev/bin/mapcache_seed)
==566==    by 0xB9A7E0D: start_thread (in /lib64/libpthread-2.15.so)
==566==    by 0x7B692CC: clone (in /lib64/libc-2.15.so)
==566==

and the longer run reports larger amounts of memory leaked for the same lines of code:

==778== 1,199,023 (586,464 direct, 612,559 indirect) bytes in 1,788 blocks are definitely lost in loss record 82 of 84
==778==    at 0x4C2ABED: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==778==    by 0xA592AB8: pj_malloc (in /usr/lib64/libproj.so.0.7.0)
==778==    by 0xA579461: pj_merc (in /usr/lib64/libproj.so.0.7.0)
==778==    by 0xA591D74: pj_init_ctx (in /usr/lib64/libproj.so.0.7.0)
==778==    by 0x7242801: msProcessProjection (in /usr/lib64/libmapserver-6.2.0.so)
==778==    by 0x7242970: msLoadProjectionString (in /usr/lib64/libmapserver-6.2.0.so)
==778==    by 0x4E62E67: _mapcache_source_mapserver_render_map (in /opt/mapcache-1.2.1-ms-6.2-dev/lib/libmapcache.so.1.2.1)
==778==    by 0x4E65860: mapcache_tileset_render_metatile (in /opt/mapcache-1.2.1-ms-6.2-dev/lib/libmapcache.so.1.2.1)
==778==    by 0x4048C8: seed_worker (in /opt/mapcache-1.2.1-ms-6.2-dev/bin/mapcache_seed)
==778==    by 0x404A59: seed_thread (in /opt/mapcache-1.2.1-ms-6.2-dev/bin/mapcache_seed)
==778==    by 0xB9A7E0D: start_thread (in /lib64/libpthread-2.15.so)
==778==    by 0x7B692CC: clone (in /lib64/libc-2.15.so)

==778== 59,070 (28,640 direct, 30,430 indirect) bytes in 1,790 blocks are definitely lost in loss record 78 of 84
==778==    at 0x4C2ABED: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==778==    by 0x725421D: msSmallMalloc (in /usr/lib64/libmapserver-6.2.0.so)
==778==    by 0x7242BBF: msLoadProjectionString (in /usr/lib64/libmapserver-6.2.0.so)
==778==    by 0x4E62E67: _mapcache_source_mapserver_render_map (in /opt/mapcache-1.2.1-ms-6.2-dev/lib/libmapcache.so.1.2.1)
==778==    by 0x4E65860: mapcache_tileset_render_metatile (in /opt/mapcache-1.2.1-ms-6.2-dev/lib/libmapcache.so.1.2.1)
==778==    by 0x4048C8: seed_worker (in /opt/mapcache-1.2.1-ms-6.2-dev/bin/mapcache_seed)
==778==    by 0x404A59: seed_thread (in /opt/mapcache-1.2.1-ms-6.2-dev/bin/mapcache_seed)
==778==    by 0xB9A7E0D: start_thread (in /lib64/libpthread-2.15.so)
==778==    by 0x7B692CC: clone (in /lib64/libc-2.15.so)
==778==

I have verified with a simple loop that msLoadProjectionString() by itself doesn't leak in the same environment, so it seems to be something that mapcache_seed does that causes the leak. i.e. the following loop doesn't leak:

        projectionObj p;
    int i=0;
        printf("Testing projectionObj leak...\n");
    msInitProjection(&p);
        for (i=0; i < 100; i++)
        {
          if (msLoadProjectionString(&p, "EPSG:900913") != 0)
            msCGIWriteError(mapserv);
        }
        msFreeProjection(&p);
        return 0;

As I wrote we will try to produce a self-contained testcase unless you (@tbonfort) have an idea of what may be causing the leak?

dmorissette commented 10 years ago

BTW, mapcache_seed is run with the following command:

mapcache_seed -c /path/to/mapcache/osm.xml -t osm-cache -e -12500000,9500000,-10000000,12000000 -n 8 -z0,15 -i "drill-down" -v
fblackburn commented 10 years ago

I was able to reproduce the leak on my system running Ubuntu 13.10. I used mapcache 1.2.1 and mapserver 6.2 with a pretty basic mapcache xml config (I used the default one and only changed the data source).

@tbonfort would you like to have a testcase?