Segmentation fault on WKB + unicode properties (Python)

richterb commented 9 years ago

We are having an issue with our Mapnik rendering code which I'll try to describe here. We are unable to reproduce it outside of our own environment, and we have a not-too-horrible workaround, so at this point we are rather looking for more ideas to pinpoint the problem, as I think our scope is still a bit too broad.

Using Mapnik's Python bindings, we are creating a MemoryDatasource in which we are adding Features one by one. Each Feature is built from a geometry object (WKT or WKB), and we add properties to it.

We are having a violent python crash (segmentation fault) in the following conditions:

we are adding geometry into features using WKB (add_geometries_from_wkb); the problem never occurs if we use WKT (add_geometries_from_wkt) instead
we are running Ubuntu 14.04; we don't get this crash under OS X
at least one of the values we are setting as properties in the features is an unicode python string. For example f['prop'] = u'aaa' may trigger the crash, while f['prop'] = 'aaa' will not.

The crash always occurs during one of the properties assignment (e.g. f['prop'] = u'...'). It never reaches the rendering step.

The crashing code is running inside an UWSGI daemon. Following our tests, it appears that in a mono-threaded single UWSGI process, the crash appears 100% of the time on a constant test dataset, while on a multithreaded setup (10 threads) it appears to crash 50% of the time. This occurs on a Mapnik built from master in the last few days.

Unfortunately we are unable to reproduce it in a stand-alone script, but I'd be happy to hear some ideas of where to look at; since the conditions of the crash are pretty specific, it may ring a bell ;)

Our code look roughly like this:

    m = mapnik.Map(256, 256)
    mapnik.load_map(m, 'ods.xml')

    feature_gen = get_feature_generator() # A generator that yields geometry+properties objects

    layer = m.layers[0]
    ds = mapnik.MemoryDatasource()

    ids = itertools.count(0)

    fields = ["ods:field1", "ods:field2", "ods:field3", "ods:field4", "ods:field5", "ods:field6"]

    for feature in feature_gen:
        mapnik_context = mapnik.Context()
        for field in fields:
            mapnik_context.push(field)

        f = mapnik.Feature(mapnik_context, ids.next())

        f.add_geometries_from_wkb(base64.decodestring(feature['geometry'])) # Using WKB 

        feature_properties = feature.get('properties', {})

        f['shape'] = feature_properties.get('shape')
        f['marker'] = feature_properties.get('marker')
        f['marker-icon'] = feature_properties.get('icon')

        ds.add_feature(f)

    # The rest of the rendering usual stuff

artemp commented 9 years ago

@richterb - Thanks for reporting. My only guess at this moment is some kind of memory corruption with add_geometries_from_wkb and adding unicode property value is just a trigger .. quite vague I know. Please, let us know if you get any more specific details or how to reproduce.

richterb commented 9 years ago

Unfortunately, our attempts to reproduce it outside of our big project don't seem to trigger it. We're probably missing something here.

A few more details after some testing:

we have the problem both on Mapnik's generated .deb (from https://launchpad.net/~mapnik/+archive/ubuntu/nightly-trunk/+packages) and from our own custom install from Github's master, on our Ubuntu 14.04.1
it seems that for a constant tile request, the amount of requests to make it crash is constant. For example, we get a first dataset (350 points with 5 properties), it crashes 100% of the time. On a second (750 points), first request works, then the second crashes. The daemon starts fresh again, and the first will work, the second crash, and so on. And on a third dataset (100 points), it crashes at exactly the twelth request.

If you have any idea of things we could do to help analysis on our crashing case, I'll happily take it. :) We'll also continue to try to reproduce it independently...

richterb commented 9 years ago

Taken from a core dump (happens at the time of the crash, so during a property assignment as described in my first post):

/usr/local/bin/uwsgi(uwsgi_backtrace+0x2e) [0x467c2e]
/usr/local/bin/uwsgi(uwsgi_segfault+0x21) [0x467ff1]
/lib/x86_64-linux-gnu/libc.so.6(+0x37000) [0x7f9c4afa3000]
/lib/x86_64-linux-gnu/libc.so.6(strlen+0x2a) [0x7f9c4aff593a]
/usr/lib/x86_64-linux-gnu/libicuuc.so.52(_ZN6icu_5213UnicodeStringC2EPKc+0x2f) [0x7f9c2b4c46bf]
/data/virtualenv/ODS/local/lib/python2.7/site-packages/mapnik/_mapnik.so(_ZN30unicode_string_from_python_str9constructEP7_objectPN5boost6python9converter30rvalue_from_python_stage1_dataE+0x6c) [0x7f9c31ee2c0c]
/data/virtualenv/ODS/local/lib/python2.7/site-packages/mapnik/_mapnik.so(_ZN5boost6python9converter8implicitIN6icu_5213UnicodeStringEN6mapnik17value_adl_barrier5valueEE9constructEP7_objectPNS1_30rvalue_from_python_stage1_dataE+0x38) [0x7f9c31ede308]
/data/virtualenv/ODS/local/lib/python2.7/site-packages/mapnik/_mapnik.so(_ZN5boost6python6detail12caller_arityILj3EE4implIPFvRN6mapnik12feature_implERKSsRKNS5_17value_adl_barrier5valueEENS0_21default_call_policiesENS_3mpl7vector4IvS7_S9_SD_EEEclEP7_objectSM_+0xcd) [0x7f9c31ed95bd]
/usr/lib/x86_64-linux-gnu/libboost_python-py27.so.1.54.0(_ZNK5boost6python7objects8function4callEP7_objectS4_+0xca) [0x7f9c30b8964a]
/usr/lib/x86_64-linux-gnu/libboost_python-py27.so.1.54.0(+0x279b8) [0x7f9c30b899b8]
/usr/lib/x86_64-linux-gnu/libboost_python-py27.so.1.54.0(_ZNK5boost6python6detail17exception_handlerclERKNS_9function0IvEE+0x43) [0x7f9c30b93ea3]
/data/virtualenv/ODS/local/lib/python2.7/site-packages/mapnik/_mapnik.so(_ZN5boost6detail8function21function_obj_invoker2INS_3_bi6bind_tIbNS_6python6detail19translate_exceptionISt13runtime_errorPFvRKS8_EEENS3_5list3INS_3argILi1EEENSF_ILi2EEENS3_5valueISC_EEEEEEbRKNS6_17exception_handlerERKNS_9function0IvEEE6invokeERNS1_15function_bufferESO_SS_+0x13) [0x7f9c31f613b3]
/usr/lib/x86_64-linux-gnu/libboost_python-py27.so.1.54.0(_ZNK5boost6python6detail17exception_handlerclERKNS_9function0IvEE+0x28) [0x7f9c30b93e88]
/data/virtualenv/ODS/local/lib/python2.7/site-packages/mapnik/_mapnik.so(_ZN5boost6detail8function21function_obj_invoker2INS_3_bi6bind_tIbNS_6python6detail19translate_exceptionIN6mapnik11value_errorEPFvRKS9_EEENS3_5list3INS_3argILi1EEENSG_ILi2EEENS3_5valueISD_EEEEEEbRKNS6_17exception_handlerERKNS_9function0IvEEE6invokeERNS1_15function_bufferESP_ST_+0x13) [0x7f9c31f61493]
/usr/lib/x86_64-linux-gnu/libboost_python-py27.so.1.54.0(_ZNK5boost6python6detail17exception_handlerclERKNS_9function0IvEE+0x28) [0x7f9c30b93e88]
/data/virtualenv/ODS/local/lib/python2.7/site-packages/mapnik/_mapnik.so(_ZN5boost6detail8function21function_obj_invoker2INS_3_bi6bind_tIbNS_6python6detail19translate_exceptionISt12out_of_rangePFvRKS8_EEENS3_5list3INS_3argILi1EEENSF_ILi2EEENS3_5valueISC_EEEEEEbRKNS6_17exception_handlerERKNS_9function0IvEEE6invokeERNS1_15function_bufferESO_SS_+0x13) [0x7f9c31f61573]
/usr/lib/x86_64-linux-gnu/libboost_python-py27.so.1.54.0(_ZNK5boost6python6detail17exception_handlerclERKNS_9function0IvEE+0x28) [0x7f9c30b93e88]
/data/virtualenv/ODS/local/lib/python2.7/site-packages/mapnik/_mapnik.so(_ZN5boost6detail8function21function_obj_invoker2INS_3_bi6bind_tIbNS_6python6detail19translate_exceptionISt9exceptionPFvRKS8_EEENS3_5list3INS_3argILi1EEENSF_ILi2EEENS3_5valueISC_EEEEEEbRKNS6_17exception_handlerERKNS_9function0IvEEE6invokeERNS1_15function_bufferESO_SS_+0x13) [0x7f9c31f61653]
/usr/lib/x86_64-linux-gnu/libboost_python-py27.so.1.54.0(_ZN5boost6python21handle_exception_implENS_9function0IvEE+0x2d) [0x7f9c30b93c6d]
/usr/lib/x86_64-linux-gnu/libboost_python-py27.so.1.54.0(+0x262c3) [0x7f9c30b882c3]
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(PyObject_Call+0x43) [0x7f9c4b45be23]
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(+0x7d30d) [0x7f9c4b3af30d]
... [ more lines about libpython, removed for clarity ]

Our boost version is 1.54.

Aquassaut commented 9 years ago

Hi,

We ran everything under valgrind, here are our results.

First, richterb said that not all requests trigger the segfault. It is totally right, but as it turns out, valgrind complains even on those requests. here are the relevant tidbits:

First error:

==10155== Invalid read of size 4
==10155==    at 0x64D6852: PyObject_Free (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==10155==    by 0x23D0587A: boost::python::api::object& boost::python::api::operator+=<char const*>(boost::python::api::object&, char const* const&) (in /usr/lib/x86_64-linux-gnu/libboost_python-py27.so.1.54.0)
==10155==    by 0x23D0432C: boost::python::objects::function::add_to_namespace(boost::python::api::object const&, char const*, boost::python::api::object const&, char const*) (in /usr/lib/x86_64-linux-gnu/libboost_python-py27.so.1.54.0)
==10155==    by 0x23D0E80B: boost::python::detail::scope_setattr_doc(char const*, boost::python::api::object const&, char const*) (in /usr/lib/x86_64-linux-gnu/libboost_python-py27.so.1.54.0)
==10155==    by 0x22B01C8E: void boost::python::detail::def_from_helper<void (*)(mapnik::Map const&, std::string const&, std::string const&), boost::python::detail::def_helper<char [507], boost::python::detail::not_specified, boost::python::detail::not_specified, boost::python::detail::not_specified> >(char const*, void (* const&)(mapnik::Map const&, std::string const&, std::string const&), boost::python::detail::def_helper<char [507], boost::python::detail::not_specified, boost::python::detail::not_specified, boost::python::detail::not_specified> const&) (in /usr/lib/pyshared/python2.7/mapnik/_mapnik.so)
==10155==    by 0x22B01C16: void boost::python::detail::def_maybe_overloads<void (*)(mapnik::Map const&, std::string const&, std::string const&), char [507]>(char const*, void (*)(mapnik::Map const&, std::string const&, std::string const&), char const (&) [507], ...) (in /usr/lib/pyshared/python2.7/mapnik/_mapnik.so)
==10155==    by 0x22AEFE39: init_module__mapnik() (in /usr/lib/pyshared/python2.7/mapnik/_mapnik.so)
==10155==    by 0x23D0DC92: boost::python::handle_exception_impl(boost::function0<void>) (in /usr/lib/x86_64-linux-gnu/libboost_python-py27.so.1.54.0)
==10155==    by 0x23D0E925: boost::python::detail::init_module(char const*, void (*)()) (in /usr/lib/x86_64-linux-gnu/libboost_python-py27.so.1.54.0)
==10155==    by 0x64EF548: _PyImport_LoadDynamicModule (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==10155==    by 0x648D538: ??? (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==10155==    by 0x648D882: PyImport_ImportModuleLevel (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==10155==  Address 0x72e7020 is 2,992 bytes inside a block of size 4,097 free'd
==10155==    at 0x4C2BDEC: free (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==10155==    by 0x640CC07: ??? (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==10155==    by 0x640D0F6: ??? (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==10155==    by 0x647B0D3: PyEval_EvalFrameEx (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==10155==    by 0x647B058: PyEval_EvalFrameEx (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==10155==    by 0x647B058: PyEval_EvalFrameEx (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==10155==    by 0x647D17C: PyEval_EvalCodeEx (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==10155==    by 0x647D461: PyEval_EvalCode (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==10155==    by 0x647DCBB: PyImport_ExecCodeModuleEx (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==10155==    by 0x648CE25: ??? (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==10155==    by 0x648D538: ??? (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==10155==    by 0x648D941: PyImport_ImportModuleLevel (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)

Second error:

==10155== Invalid read of size 4
==10155==    at 0x64D6852: PyObject_Free (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==10155==    by 0x23D11836: boost::python::api::operator+=(boost::python::api::object&, boost::python::api::object const&) (in /usr/lib/x86_64-linux-gnu/libboost_python-py27.so.1.54.0)
==10155==    by 0x23D042A7: boost::python::objects::function::add_to_namespace(boost::python::api::object const&, char const*, boost::python::api::object const&, char const*) (in /usr/lib/x86_64-linux-gnu/libboost_python-py27.so.1.54.0)
==10155==    by 0x23D0E80B: boost::python::detail::scope_setattr_doc(char const*, boost::python::api::object const&, char const*) (in /usr/lib/x86_64-linux-gnu/libboost_python-py27.so.1.54.0)
==10155==    by 0x22B01C8E: void boost::python::detail::def_from_helper<void (*)(mapnik::Map const&, std::string const&, std::string const&), boost::python::detail::def_helper<char [507], boost::python::detail::not_specified, boost::python::detail::not_specified, boost::python::detail::not_specified> >(char const*, void (* const&)(mapnik::Map const&, std::string const&, std::string const&), boost::python::detail::def_helper<char [507], boost::python::detail::not_specified, boost::python::detail::not_specified, boost::python::detail::not_specified> const&) (in /usr/lib/pyshared/python2.7/mapnik/_mapnik.so)
==10155==    by 0x22B01C16: void boost::python::detail::def_maybe_overloads<void (*)(mapnik::Map const&, std::string const&, std::string const&), char [507]>(char const*, void (*)(mapnik::Map const&, std::string const&, std::string const&), char const (&) [507], ...) (in /usr/lib/pyshared/python2.7/mapnik/_mapnik.so)
==10155==    by 0x22AEFE39: init_module__mapnik() (in /usr/lib/pyshared/python2.7/mapnik/_mapnik.so)
==10155==    by 0x23D0DC92: boost::python::handle_exception_impl(boost::function0<void>) (in /usr/lib/x86_64-linux-gnu/libboost_python-py27.so.1.54.0)
==10155==    by 0x23D0E925: boost::python::detail::init_module(char const*, void (*)()) (in /usr/lib/x86_64-linux-gnu/libboost_python-py27.so.1.54.0)
==10155==    by 0x64EF548: _PyImport_LoadDynamicModule (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==10155==    by 0x648D538: ??? (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==10155==    by 0x648D882: PyImport_ImportModuleLevel (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==10155==  Address 0x1bdee020 is not stack'd, malloc'd or (recently) free'd

Third error:

==10155== Invalid read of size 4
==10155==    at 0x64D6852: PyObject_Free (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==10155==    by 0x6477DE9: PyEval_EvalFrameEx (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==10155==    by 0x647D17C: PyEval_EvalCodeEx (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==10155==    by 0x647ADD7: PyEval_EvalFrameEx (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==10155==    by 0x647D17C: PyEval_EvalCodeEx (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==10155==    by 0x647ADD7: PyEval_EvalFrameEx (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==10155==    by 0x647D17C: PyEval_EvalCodeEx (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==10155==    by 0x647D3E4: ??? (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==10155==    by 0x6444E22: PyObject_Call (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==10155==    by 0x64773B0: PyEval_EvalFrameEx (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==10155==    by 0x647D17C: PyEval_EvalCodeEx (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==10155==    by 0x647D3E4: ??? (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==10155==  Address 0x2a3f6020 is 21,472 bytes inside a block of size 23,048 free'd
==10155==    at 0x4C2C83C: operator delete[](void*) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==10155==    by 0x238096D8: _ZN6mapnik25render_markers_symbolizerINS_6detail34vector_markers_rasterizer_dispatchINS_3svg16svg_renderer_aggINS3_12path_adapterINS3_18vertex_stl_adapterISt6vectorIN3agg11vertex_baseIdEESaISA_EEEEEENS8_11pod_bvectorINS3_15path_attributesELj6EEENS8_26renderer_scanline_aa_solidINS8_13renderer_baseINS8_24pixfmt_custom_blend_rgbaINS8_24comp_op_adaptor_rgba_preINS8_6rgba8TINS8_6linearEEENS8_10order_rgbaEEENS8_13row_ptr_cacheIhEEEEEEEEST_EENS_25label_collision_detector4ESt5tupleIJRSS_RNS_10rasterizerERNS_8image_32EEEEENS1_34raster_markers_rasterizer_dispatchISX_S14_EENS_15renderer_commonES14_EEvRKNS_18markers_symbolizerERNS_12feature_implERKNS_14proj_transformERKT1_RKNS_5box2dIdEERKT2_ (in /usr/lib/libmapnik.so.3.0.0)
==10155==    by 0x23808C98: mapnik::agg_renderer<mapnik::image_32, mapnik::label_collision_detector4>::process(mapnik::markers_symbolizer const&, mapnik::feature_impl&, mapnik::proj_transform const&) (in /usr/lib/libmapnik.so.3.0.0)
==10155==    by 0x233CEC33: mapnik::feature_style_processor<mapnik::agg_renderer<mapnik::image_32, mapnik::label_collision_detector4> >::render_style(mapnik::agg_renderer<mapnik::image_32, mapnik::label_collision_detector4>&, mapnik::feature_type_style const*, mapnik::rule_cache const&, std::shared_ptr<mapnik::Featureset>, mapnik::proj_transform const&) (in /usr/lib/libmapnik.so.3.0.0)
==10155==    by 0x233CC4C5: mapnik::feature_style_processor<mapnik::agg_renderer<mapnik::image_32, mapnik::label_collision_detector4> >::render_material(mapnik::layer_rendering_material&, mapnik::agg_renderer<mapnik::image_32, mapnik::label_collision_detector4>&) (in /usr/lib/libmapnik.so.3.0.0)
==10155==    by 0x233CA4C9: mapnik::feature_style_processor<mapnik::agg_renderer<mapnik::image_32, mapnik::label_collision_detector4> >::apply(double) (in /usr/lib/libmapnik.so.3.0.0)
==10155==    by 0x22AEBD3B: render(mapnik::Map const&, mapnik::image_32&, double, unsigned int, unsigned int) (in /usr/lib/pyshared/python2.7/mapnik/_mapnik.so)
==10155==    by 0x22AFFA54: boost::python::detail::caller_arity<2u>::impl<void (*)(mapnik::Map const&, mapnik::image_32&), boost::python::default_call_policies, boost::mpl::vector3<void, mapnik::Map const&, mapnik::image_32&> >::operator()(_object*, _object*) (in /usr/lib/pyshared/python2.7/mapnik/_mapnik.so)
==10155==    by 0x23D03649: boost::python::objects::function::call(_object*, _object*) const (in /usr/lib/x86_64-linux-gnu/libboost_python-py27.so.1.54.0)
==10155==    by 0x23D039B7: ??? (in /usr/lib/x86_64-linux-gnu/libboost_python-py27.so.1.54.0)
==10155==    by 0x23D0DEA2: boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const (in /usr/lib/x86_64-linux-gnu/libboost_python-py27.so.1.54.0)
==10155==    by 0x22B023B2: boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<std::runtime_error, void (*)(std::runtime_error const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(std::runtime_error const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&) (in /usr/lib/pyshared/python2.7/mapnik/_mapnik.so)

Fourth error:

==10155== Invalid read of size 4
==10155==    at 0x64D6852: PyObject_Free (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==10155==    by 0x64D7219: ??? (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==10155==    by 0x649AD39: ??? (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==10155==    by 0x649ABDE: ??? (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==10155==    by 0x649ABDE: ??? (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==10155==    by 0x649AE61: ??? (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==10155==    by 0x647CED9: PyEval_EvalCodeEx (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==10155==    by 0x647ADD7: PyEval_EvalFrameEx (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==10155==    by 0x647D17C: PyEval_EvalCodeEx (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==10155==    by 0x647D3E4: ??? (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==10155==    by 0x6444E22: PyObject_Call (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==10155==    by 0x64773B0: PyEval_EvalFrameEx (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==10155==  Address 0x8808020 is 144 bytes inside a block of size 512 free'd
==10155==    at 0x4C2C83C: operator delete[](void*) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==10155==    by 0x22A89DBF: void mapnik::grid2utf<mapnik::hit_grid<long long> >(mapnik::hit_grid<long long> const&, boost::python::list&, std::vector<mapnik::hit_grid<long long>::lookup_type, std::allocator<mapnik::hit_grid<long long>::lookup_type> >&, unsigned int) (in /usr/lib/pyshared/python2.7/mapnik/_mapnik.so)
==10155==    by 0x22A858F3: void mapnik::grid_encode_utf<mapnik::hit_grid<long long> >(mapnik::hit_grid<long long> const&, boost::python::dict&, bool, unsigned int) (in /usr/lib/pyshared/python2.7/mapnik/_mapnik.so)
==10155==    by 0x22A85655: boost::python::dict mapnik::grid_encode<mapnik::hit_grid<long long> >(mapnik::hit_grid<long long> const&, std::string const&, bool, unsigned int) (in /usr/lib/pyshared/python2.7/mapnik/_mapnik.so)
==10155==    by 0x22AA01F6: boost::python::detail::caller_arity<4u>::impl<boost::python::dict (*)(mapnik::hit_grid<long long> const&, std::string const&, bool, unsigned int), boost::python::default_call_policies, boost::mpl::vector5<boost::python::dict, mapnik::hit_grid<long long> const&, std::string const&, bool, unsigned int> >::operator()(_object*, _object*) (in /usr/lib/pyshared/python2.7/mapnik/_mapnik.so)
==10155==    by 0x23D03649: boost::python::objects::function::call(_object*, _object*) const (in /usr/lib/x86_64-linux-gnu/libboost_python-py27.so.1.54.0)
==10155==    by 0x23D039B7: ??? (in /usr/lib/x86_64-linux-gnu/libboost_python-py27.so.1.54.0)
==10155==    by 0x23D0DEA2: boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const (in /usr/lib/x86_64-linux-gnu/libboost_python-py27.so.1.54.0)
==10155==    by 0x22B023B2: boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<std::runtime_error, void (*)(std::runtime_error const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(std::runtime_error const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&) (in /usr/lib/pyshared/python2.7/mapnik/_mapnik.so)
==10155==    by 0x23D0DE87: boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const (in /usr/lib/x86_64-linux-gnu/libboost_python-py27.so.1.54.0)
==10155==    by 0x22B02492: boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<mapnik::value_error, void (*)(mapnik::value_error const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(mapnik::value_error const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&) (in /usr/lib/pyshared/python2.7/mapnik/_mapnik.so)
==10155==    by 0x23D0DE87: boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const (in /usr/lib/x86_64-linux-gnu/libboost_python-py27.so.1.54.0)

Finally, on the next request, which produces a segv the error is the following:

==10155== Invalid read of size 1   
==10155==    at 0x4C2E0E2: strlen (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==10155==    by 0x252666BE: icu_52::UnicodeString::UnicodeString(char const*) (in /usr/lib/x86_64-linux-gnu/libicuuc.so.52.1)
==10155==    by 0x22A83C0B: unicode_string_from_python_str::construct(_object*, boost::python::converter::rvalue_from_python_stage1_data*) (in /usr/lib/pyshared/python2.7/mapnik/_mapnik.so)
==10155==    by 0x22A7F307: boost::python::converter::implicit<icu_52::UnicodeString, mapnik::value_adl_barrier::value>::construct(_object*, boost::python::converter::rvalue_from_python_stage1_data*) (in /usr/lib/pyshared/python2.7/mapnik/_mapnik.so)
==10155==    by 0x22A7A5BC: boost::python::detail::caller_arity<3u>::impl<void (*)(mapnik::feature_impl&, std::string const&, mapnik::value_adl_barrier::value const&), boost::python::default_call_policies, boost::mpl::vector4<void, mapnik::feature_impl&, std::string const&, mapnik::value_adl_barrier::value const&> >::operator()(_object*, _object*) (in /usr/lib/pyshared/python2.7/mapnik/_mapnik.so)
==10155==    by 0x23D03649: boost::python::objects::function::call(_object*, _object*) const (in /usr/lib/x86_64-linux-gnu/libboost_python-py27.so.1.54.0)
==10155==    by 0x23D039B7: ??? (in /usr/lib/x86_64-linux-gnu/libboost_python-py27.so.1.54.0)
==10155==    by 0x23D0DEA2: boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const (in /usr/lib/x86_64-linux-gnu/libboost_python-py27.so.1.54.0)
==10155==    by 0x22B023B2: boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<std::runtime_error, void (*)(std::runtime_error const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(std::runtime_error const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&) (in /usr/lib/pyshared/python2.7/mapnik/_mapnik.so)
==10155==    by 0x23D0DE87: boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const (in /usr/lib/x86_64-linux-gnu/libboost_python-py27.so.1.54.0)
==10155==    by 0x22B02492: boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<mapnik::value_error, void (*)(mapnik::value_error const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(mapnik::value_error const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&) (in /usr/lib/pyshared/python2.7/mapnik/_mapnik.so)
==10155==    by 0x23D0DE87: boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const (in /usr/lib/x86_64-linux-gnu/libboost_python-py27.so.1.54.0)
==10155==  Address 0x2b8ed054 is not stack'd, malloc'd or (recently) free'd

Please let us know if we can be of assistance in any way to track this bug down. Thank you.

springmeyer commented 9 years ago

@richterb - can you post the code for get_feature_generator? I see it is creating mapnik.Feature objects, but then you modify them later inside for feature in feature_gen:. I wonder if you have multiple threads modifying the same underlying features (which is bound is crash)?

richterb commented 9 years ago

Woops, my "shortened" code example has been shortened a bit too much. get_feature_generator is responsible for returning features in the sense of "geographical features", it does not touch Mapnik at all. We create a new mapnik.Feature each time directly in the loop.

Our code actually looks like this (I updated the original post as well):

    for feature in feature_gen:
        mapnik_context = mapnik.Context()
        for field in ods_fields:
            mapnik_context.push(field)

        f = mapnik.Feature(mapnik_context, ids.next())

        f.add_geometries_from_wkb(base64.decodestring(feature['geometry']))

springmeyer commented 9 years ago

Okay, thanks. BTW, you only need to create the mapnik_context once and you can reuse it for each feature.

mapnik / mapnik-support

Segmentation fault on WKB + unicode properties (Python) #31