Error: Vector subscript out of range

[DiabloRusso]

Hi! I'm trying mapnik (branch v3.0.x) and mapnik-vector-tile (v1.2.2). I build x64 debug configuration for Windows. My test code:

mapnik::datasource_cache::instance().register_datasource(mapnik_dir + "/mapnik/input/shape.input");
mapnik::Map m(256, 256);
load_map(m, "style.xml");   
m.zoom_all();
mapnik::vector_tile_impl::processor ren(m);
mapnik::vector_tile_impl::merc_tile tile(0, 0, 0, 1024);
ren.update_tile(tile);

My simplified mapnik style:

<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE Map[]>
<Map srs="+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0.0 +k=1.0 +units=m +nadgrids=@null +wktext +no_defs +over" background-color="#b5d0d0">

<Parameters>
  <Parameter name="scale">1</Parameter>
  <Parameter name="metatile">2</Parameter>
  <Parameter name="name"><![CDATA[OpenStreetMap Carto]]></Parameter>
  <Parameter name="description"><![CDATA[A general-purpose OpenStreetMap mapnik style, in CartoCSS]]></Parameter>
  <Parameter name="bounds">-180,-85.05112877980659,180,85.05112877980659</Parameter>
  <Parameter name="center">0,0,4</Parameter>
  <Parameter name="format">png</Parameter>
  <Parameter name="minzoom">0</Parameter>
  <Parameter name="maxzoom">22</Parameter>
</Parameters>
<Style name="world" filter-mode="first">
  <Rule>
    <MinScaleDenominator>750000</MinScaleDenominator>
    <PolygonSymbolizer fill="#f2efe9" />
  </Rule>
</Style>
<Layer name="world"
  minimum-scale-denominator="750000"
  srs="+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0.0 +k=1.0 +units=m +nadgrids=@null +wktext +no_defs +over">
    <StyleName>world</StyleName>
    <Datasource>
       <Parameter name="file"><![CDATA[Path to\simplified_land_polygons.shp]]></Parameter>
       <Parameter name="type"><![CDATA[shape]]></Parameter>
    </Datasource>
  </Layer>

</Map>

I get an error like this? call stack:

>   test.exe!std::vector<ClipperLib::OutRec * __ptr64,std::allocator<ClipperLib::OutRec * __ptr64> >::operator[](unsigned __int64 _Pos) Line 1237   C++
    test.exe!ClipperLib::Clipper::AppendPolygon(ClipperLib::TEdge * e1, ClipperLib::TEdge * e2) Line 2735   C++
    test.exe!ClipperLib::Clipper::AddLocalMaxPoly(ClipperLib::TEdge * e1, ClipperLib::TEdge * e2, const mapnik::geometry::point<__int64> & Pt) Line 2267    C++
    test.exe!ClipperLib::Clipper::ProcessHorizontal(ClipperLib::TEdge * horzEdge) Line 3103 C++
    test.exe!ClipperLib::Clipper::ProcessHorizontals() Line 2881    C++
    test.exe!ClipperLib::Clipper::ProcessEdgesAtTopOfScanbeam(const __int64 topY) Line 3434 C++
    test.exe!ClipperLib::Clipper::ExecuteInternal() Line 1952   C++
    test.exe!ClipperLib::Clipper::Execute(ClipperLib::ClipType clipType, ClipperLib::PolyTree & polytree, ClipperLib::PolyFillType subjFillType, ClipperLib::PolyFillType clipFillType) Line 1907   C++
    test.exe!mapnik::vector_tile_impl::geometry_clipper<mapnik::vector_tile_impl::geometry_to_feature_pbf_visitor>::operator()(mapnik::geometry::polygon<__int64,mapnik::geometry::rings_container> & geom) Line 292    C++
    test.exe!mapnik::vector_tile_impl::transform_visitor<mapnik::vector_tile_impl::vector_tile_strategy,mapnik::vector_tile_impl::geometry_clipper<mapnik::vector_tile_impl::geometry_to_feature_pbf_visitor> >::operator()(const mapnik::geometry::polygon<double,mapnik::geometry::rings_container> & geom) Line 238  C++
    test.exe!mapbox::util::detail::dispatcher<mapnik::vector_tile_impl::transform_visitor<mapnik::vector_tile_impl::vector_tile_strategy,mapnik::vector_tile_impl::geometry_clipper<mapnik::vector_tile_impl::geometry_to_feature_pbf_visitor> > & __ptr64,mapnik::geometry::geometry<double>,void,mapnik::geometry::polygon<double,mapnik::geometry::rings_container>,mapnik::geometry::multi_point<double>,mapnik::geometry::multi_line_string<double>,mapnik::geometry::multi_polygon<double>,mapnik::geometry::geometry_collection<double> >::apply_const(const mapnik::geometry::geometry<double> & v, mapnik::vector_tile_impl::transform_visitor<mapnik::vector_tile_impl::vector_tile_strategy,mapnik::vector_tile_impl::geometry_clipper<mapnik::vector_tile_impl::geometry_to_feature_pbf_visitor> > & f) Line 313    C++
    test.exe!mapbox::util::detail::dispatcher<mapnik::vector_tile_impl::transform_visitor<mapnik::vector_tile_impl::vector_tile_strategy,mapnik::vector_tile_impl::geometry_clipper<mapnik::vector_tile_impl::geometry_to_feature_pbf_visitor> > & __ptr64,mapnik::geometry::geometry<double>,void,mapnik::geometry::line_string<double>,mapnik::geometry::polygon<double,mapnik::geometry::rings_container>,mapnik::geometry::multi_point<double>,mapnik::geometry::multi_line_string<double>,mapnik::geometry::multi_polygon<double>,mapnik::geometry::geometry_collection<double> >::apply_const(const mapnik::geometry::geometry<double> & v, mapnik::vector_tile_impl::transform_visitor<mapnik::vector_tile_impl::vector_tile_strategy,mapnik::vector_tile_impl::geometry_clipper<mapnik::vector_tile_impl::geometry_to_feature_pbf_visitor> > & f) Line 319  C++
    test.exe!mapbox::util::detail::dispatcher<mapnik::vector_tile_impl::transform_visitor<mapnik::vector_tile_impl::vector_tile_strategy,mapnik::vector_tile_impl::geometry_clipper<mapnik::vector_tile_impl::geometry_to_feature_pbf_visitor> > & __ptr64,mapnik::geometry::geometry<double>,void,mapnik::geometry::point<double>,mapnik::geometry::line_string<double>,mapnik::geometry::polygon<double,mapnik::geometry::rings_container>,mapnik::geometry::multi_point<double>,mapnik::geometry::multi_line_string<double>,mapnik::geometry::multi_polygon<double>,mapnik::geometry::geometry_collection<double> >::apply_const(const mapnik::geometry::geometry<double> & v, mapnik::vector_tile_impl::transform_visitor<mapnik::vector_tile_impl::vector_tile_strategy,mapnik::vector_tile_impl::geometry_clipper<mapnik::vector_tile_impl::geometry_to_feature_pbf_visitor> > & f) Line 319  C++
    test.exe!mapbox::util::detail::dispatcher<mapnik::vector_tile_impl::transform_visitor<mapnik::vector_tile_impl::vector_tile_strategy,mapnik::vector_tile_impl::geometry_clipper<mapnik::vector_tile_impl::geometry_to_feature_pbf_visitor> > & __ptr64,mapnik::geometry::geometry<double>,void,mapnik::geometry::geometry_empty,mapnik::geometry::point<double>,mapnik::geometry::line_string<double>,mapnik::geometry::polygon<double,mapnik::geometry::rings_container>,mapnik::geometry::multi_point<double>,mapnik::geometry::multi_line_string<double>,mapnik::geometry::multi_polygon<double>,mapnik::geometry::geometry_collection<double> >::apply_const(const mapnik::geometry::geometry<double> & v, mapnik::vector_tile_impl::transform_visitor<mapnik::vector_tile_impl::vector_tile_strategy,mapnik::vector_tile_impl::geometry_clipper<mapnik::vector_tile_impl::geometry_to_feature_pbf_visitor> > & f) Line 319 C++
    test.exe!mapbox::util::variant<mapnik::geometry::geometry_empty,mapnik::geometry::point<double>,mapnik::geometry::line_string<double>,mapnik::geometry::polygon<double,mapnik::geometry::rings_container>,mapnik::geometry::multi_point<double>,mapnik::geometry::multi_line_string<double>,mapnik::geometry::multi_polygon<double>,mapnik::geometry::geometry_collection<double> >::visit<mapnik::vector_tile_impl::transform_visitor<mapnik::vector_tile_impl::vector_tile_strategy,mapnik::vector_tile_impl::geometry_clipper<mapnik::vector_tile_impl::geometry_to_feature_pbf_visitor> > & __ptr64,mapnik::geometry::geometry<double>,void>(const mapnik::geometry::geometry<double> & v, mapnik::vector_tile_impl::transform_visitor<mapnik::vector_tile_impl::vector_tile_strategy,mapnik::vector_tile_impl::geometry_clipper<mapnik::vector_tile_impl::geometry_to_feature_pbf_visitor> > & f) Line 865 C++
    test.exe!mapnik::util::apply_visitor<mapnik::vector_tile_impl::transform_visitor<mapnik::vector_tile_impl::vector_tile_strategy,mapnik::vector_tile_impl::geometry_clipper<mapnik::vector_tile_impl::geometry_to_feature_pbf_visitor> > & __ptr64,mapnik::geometry::geometry<double> >(mapnik::vector_tile_impl::transform_visitor<mapnik::vector_tile_impl::vector_tile_strategy,mapnik::vector_tile_impl::geometry_clipper<mapnik::vector_tile_impl::geometry_to_feature_pbf_visitor> > & f, const mapnik::geometry::geometry<double> & v) Line 43    C++
    test.exe!mapnik::vector_tile_impl::detail::create_geom_layer(mapnik::vector_tile_impl::tile_layer & layer, double simplify_distance, double area_threshold, mapnik::vector_tile_impl::polygon_fill_type fill_type, bool strictly_simple, bool multi_polygon_union, bool process_all_rings) Line 161 C++
    test.exe!mapnik::vector_tile_impl::processor::update_tile(mapnik::vector_tile_impl::tile & t, double scale_denom, int offset_x, int offset_y) Line 311  C++
    test.exe!main(int argc, char * * argv) Line 43  C++

This is a bug or am I doing something wrong? (ClipperLib is here) Thanx!

[flippmoke]

I am not 100% certain with out digging deep into this problem -- and as the bug is within the ClipperLib (which we are in the process of replacing), I am honestly not really tempted to currently dig into this problem too deep.

Can you first try with some other data --

Also I would use the default extents as well here:

mapnik::vector_tile_impl::merc_tile tile(0, 0, 0, 1024);

instead just do:

mapnik::vector_tile_impl::merc_tile tile(0, 0, 0);

[flippmoke]

BTW - this is the replacement for the Angus clipper library on a branch -- using wagyu. You might try it out.

[DiabloRusso]

Thanx for answer! I try wagyu branch and get a similar error with extent=1024 and default extent too:

    msvcp140d.dll!000007fee21e0b46()    Unknown
    test.exe!std::_Vector_const_iterator<std::_Vector_val<std::_Simple_types<mapbox::geometry::wagyu::edge<__int64> > > >::operator++() Line 102    C++
    test.exe!std::_Vector_iterator<std::_Vector_val<std::_Simple_types<mapbox::geometry::wagyu::edge<__int64> > > >::operator++() Line 336  C++
    test.exe!mapbox::geometry::wagyu::next_edge_in_bound<__int64>(std::_List_iterator<std::_List_val<std::_List_simple_types<mapbox::geometry::wagyu::bound<__int64> *> > > & bnd, std::priority_queue<__int64,std::vector<__int64,std::allocator<__int64> >,std::less<__int64> > & scanbeam) Line 165  C++
>   test.exe!mapbox::geometry::wagyu::process_hot_pixel_edges_at_top_of_scanbeam<__int64>(__int64 top_y, std::priority_queue<__int64,std::vector<__int64,std::allocator<__int64> >,std::less<__int64> > & scanbeam, std::list<mapbox::geometry::wagyu::bound<__int64> *,std::allocator<mapbox::geometry::wagyu::bound<__int64> *> > & active_bounds, mapbox::geometry::wagyu::ring_manager<__int64> & rings) Line 99    C++
    test.exe!mapbox::geometry::wagyu::build_hot_pixels<__int64>(std::deque<mapbox::geometry::wagyu::local_minimum<__int64>,std::allocator<mapbox::geometry::wagyu::local_minimum<__int64> > > & minima_list, mapbox::geometry::wagyu::ring_manager<__int64> & rings) Line 166   C++
    test.exe!mapbox::geometry::wagyu::wagyu<__int64>::execute(mapbox::geometry::wagyu::clip_type cliptype, mapbox::geometry::multi_polygon<__int64,std::vector> & solution, mapbox::geometry::wagyu::fill_type subject_fill_type, mapbox::geometry::wagyu::fill_type clip_fill_type) Line 124   C++
    test.exe!mapnik::vector_tile_impl::geometry_clipper<mapnik::vector_tile_impl::geometry_to_feature_pbf_visitor>::operator()(mapbox::geometry::polygon<__int64,std::vector> & geom) Line 240  C++
    test.exe!mapnik::vector_tile_impl::transform_visitor<mapnik::vector_tile_impl::vector_tile_strategy,mapnik::vector_tile_impl::geometry_clipper<mapnik::vector_tile_impl::geometry_to_feature_pbf_visitor> >::operator()(const mapnik::geometry::polygon<double,mapnik::geometry::rings_container> & geom) Line 252  C++
    test.exe!mapbox::util::detail::dispatcher<mapnik::vector_tile_impl::transform_visitor<mapnik::vector_tile_impl::vector_tile_strategy,mapnik::vector_tile_impl::geometry_clipper<mapnik::vector_tile_impl::geometry_to_feature_pbf_visitor> > & __ptr64,mapnik::geometry::geometry<double>,void,mapnik::geometry::polygon<double,mapnik::geometry::rings_container>,mapnik::geometry::multi_point<double>,mapnik::geometry::multi_line_string<double>,mapnik::geometry::multi_polygon<double>,mapnik::geometry::geometry_collection<double> >::apply_const(const mapnik::geometry::geometry<double> & v, mapnik::vector_tile_impl::transform_visitor<mapnik::vector_tile_impl::vector_tile_strategy,mapnik::vector_tile_impl::geometry_clipper<mapnik::vector_tile_impl::geometry_to_feature_pbf_visitor> > & f) Line 313    C++
    test.exe!mapbox::util::detail::dispatcher<mapnik::vector_tile_impl::transform_visitor<mapnik::vector_tile_impl::vector_tile_strategy,mapnik::vector_tile_impl::geometry_clipper<mapnik::vector_tile_impl::geometry_to_feature_pbf_visitor> > & __ptr64,mapnik::geometry::geometry<double>,void,mapnik::geometry::line_string<double>,mapnik::geometry::polygon<double,mapnik::geometry::rings_container>,mapnik::geometry::multi_point<double>,mapnik::geometry::multi_line_string<double>,mapnik::geometry::multi_polygon<double>,mapnik::geometry::geometry_collection<double> >::apply_const(const mapnik::geometry::geometry<double> & v, mapnik::vector_tile_impl::transform_visitor<mapnik::vector_tile_impl::vector_tile_strategy,mapnik::vector_tile_impl::geometry_clipper<mapnik::vector_tile_impl::geometry_to_feature_pbf_visitor> > & f) Line 319  C++
    test.exe!mapbox::util::detail::dispatcher<mapnik::vector_tile_impl::transform_visitor<mapnik::vector_tile_impl::vector_tile_strategy,mapnik::vector_tile_impl::geometry_clipper<mapnik::vector_tile_impl::geometry_to_feature_pbf_visitor> > & __ptr64,mapnik::geometry::geometry<double>,void,mapnik::geometry::point<double>,mapnik::geometry::line_string<double>,mapnik::geometry::polygon<double,mapnik::geometry::rings_container>,mapnik::geometry::multi_point<double>,mapnik::geometry::multi_line_string<double>,mapnik::geometry::multi_polygon<double>,mapnik::geometry::geometry_collection<double> >::apply_const(const mapnik::geometry::geometry<double> & v, mapnik::vector_tile_impl::transform_visitor<mapnik::vector_tile_impl::vector_tile_strategy,mapnik::vector_tile_impl::geometry_clipper<mapnik::vector_tile_impl::geometry_to_feature_pbf_visitor> > & f) Line 319  C++
    test.exe!mapbox::util::detail::dispatcher<mapnik::vector_tile_impl::transform_visitor<mapnik::vector_tile_impl::vector_tile_strategy,mapnik::vector_tile_impl::geometry_clipper<mapnik::vector_tile_impl::geometry_to_feature_pbf_visitor> > & __ptr64,mapnik::geometry::geometry<double>,void,mapnik::geometry::geometry_empty,mapnik::geometry::point<double>,mapnik::geometry::line_string<double>,mapnik::geometry::polygon<double,mapnik::geometry::rings_container>,mapnik::geometry::multi_point<double>,mapnik::geometry::multi_line_string<double>,mapnik::geometry::multi_polygon<double>,mapnik::geometry::geometry_collection<double> >::apply_const(const mapnik::geometry::geometry<double> & v, mapnik::vector_tile_impl::transform_visitor<mapnik::vector_tile_impl::vector_tile_strategy,mapnik::vector_tile_impl::geometry_clipper<mapnik::vector_tile_impl::geometry_to_feature_pbf_visitor> > & f) Line 319 C++
    test.exe!mapbox::util::variant<mapnik::geometry::geometry_empty,mapnik::geometry::point<double>,mapnik::geometry::line_string<double>,mapnik::geometry::polygon<double,mapnik::geometry::rings_container>,mapnik::geometry::multi_point<double>,mapnik::geometry::multi_line_string<double>,mapnik::geometry::multi_polygon<double>,mapnik::geometry::geometry_collection<double> >::visit<mapnik::vector_tile_impl::transform_visitor<mapnik::vector_tile_impl::vector_tile_strategy,mapnik::vector_tile_impl::geometry_clipper<mapnik::vector_tile_impl::geometry_to_feature_pbf_visitor> > & __ptr64,mapnik::geometry::geometry<double>,void>(const mapnik::geometry::geometry<double> & v, mapnik::vector_tile_impl::transform_visitor<mapnik::vector_tile_impl::vector_tile_strategy,mapnik::vector_tile_impl::geometry_clipper<mapnik::vector_tile_impl::geometry_to_feature_pbf_visitor> > & f) Line 865 C++
    test.exe!mapnik::util::apply_visitor<mapnik::vector_tile_impl::transform_visitor<mapnik::vector_tile_impl::vector_tile_strategy,mapnik::vector_tile_impl::geometry_clipper<mapnik::vector_tile_impl::geometry_to_feature_pbf_visitor> > & __ptr64,mapnik::geometry::geometry<double> >(mapnik::vector_tile_impl::transform_visitor<mapnik::vector_tile_impl::vector_tile_strategy,mapnik::vector_tile_impl::geometry_clipper<mapnik::vector_tile_impl::geometry_to_feature_pbf_visitor> > & f, const mapnik::geometry::geometry<double> & v) Line 43    C++
    test.exe!mapnik::vector_tile_impl::detail::create_geom_layer(mapnik::vector_tile_impl::tile_layer & layer, double simplify_distance, double area_threshold, mapnik::vector_tile_impl::polygon_fill_type fill_type, bool strictly_simple, bool multi_polygon_union, bool process_all_rings) Line 167 C++
    test.exe!mapnik::vector_tile_impl::processor::update_tile(mapnik::vector_tile_impl::tile & t, double scale_denom, int offset_x, int offset_y) Line 316  C++
    test.exe!main(int argc, char * * argv) Line 43  C++

I used the master branches of projects wagyu and geometry.hpp. Thanx!

[flippmoke]

@DiabloRusso that is really interesting -- is this a really massive polygon? Could you put together a good test case for this so that I might be able to repeat the problem?

[DiabloRusso]

This my complete test project! Thanks for help!

[flippmoke]

@DiabloRusso okay, I have attempted to recreate this on my machine, but I am not able to do so currently. It all seems to run just fine. Unfortunately I am using OS X and not a windows machine so we might be having slight differences between the two.

[DiabloRusso]

@flippmoke Could you try my xml style for generating a vector tiles on OS X? Maybe this error is independent of the OS and you reproduce her?

[flippmoke]

@DiabloRusso I was using your xml file and data for my test

[DiabloRusso]

@flippmoke please let me know when you get the test result. Thanx!

[flippmoke]

@DiabloRusso I used your test data and files, I was unable to reproduce your results.

[DiabloRusso]

@flippmoke maybe I do not use the correct version of wagyu and geometry.hpp?

[wilhelmberg]

Took a look at the provided test case:

Using the "empty" style.xml that comes with the package:

<Map>
<Style name="style">
    <Rule>
        <MarkersSymbolizer fill="red" />
    </Rule>
</Style>
</Map>

• Visual Studio 2015 Update 3: compiles and runs the test case without any exceptions.
• Visual Studio 2017 RC.4 completely crashes :bomb: :fire: already at mapnik::Map m(256, 256);. Maybe because of some runtime differences (VS2015 vs VS2017)?

---

Using VS2015 with the style.xml form the first post above (referencing simplified_land_polygons.shp) exception occurs on ren.update_tile(tile);:

    msvcp140d.dll!00007ffe6c7ffe06()    Unknown
>   test.exe!std::_Vector_const_iterator<std::_Vector_val<std::_Simple_types<mapbox::geometry::wagyu::edge<__int64> > > >::operator++() Line 103    C++
    test.exe!std::_Vector_iterator<std::_Vector_val<std::_Simple_types<mapbox::geometry::wagyu::edge<__int64> > > >::operator++() Line 342  C++
    test.exe!mapbox::geometry::wagyu::next_edge_in_bound<__int64>(std::_List_iterator<std::_List_val<std::_List_simple_types<mapbox::geometry::wagyu::bound<__int64> *> > > & bnd, std::priority_queue<__int64,std::vector<__int64,std::allocator<__int64> >,std::less<__int64> > & scanbeam) Line 165  C++
    test.exe!mapbox::geometry::wagyu::process_hot_pixel_edges_at_top_of_scanbeam<__int64>(__int64 top_y, std::priority_queue<__int64,std::vector<__int64,std::allocator<__int64> >,std::less<__int64> > & scanbeam, std::list<mapbox::geometry::wagyu::bound<__int64> *,std::allocator<mapbox::geometry::wagyu::bound<__int64> *> > & active_bounds, mapbox::geometry::wagyu::ring_manager<__int64> & rings) Line 99    C++
    test.exe!mapbox::geometry::wagyu::build_hot_pixels<__int64>(std::deque<mapbox::geometry::wagyu::local_minimum<__int64>,std::allocator<mapbox::geometry::wagyu::local_minimum<__int64> > > & minima_list, mapbox::geometry::wagyu::ring_manager<__int64> & rings) Line 166   C++
    test.exe!mapbox::geometry::wagyu::wagyu<__int64>::execute(mapbox::geometry::wagyu::clip_type cliptype, mapbox::geometry::multi_polygon<__int64,std::vector> & solution, mapbox::geometry::wagyu::fill_type subject_fill_type, mapbox::geometry::wagyu::fill_type clip_fill_type) Line 124   C++
    test.exe!mapnik::vector_tile_impl::geometry_clipper<mapnik::vector_tile_impl::geometry_to_feature_pbf_visitor>::operator()(mapbox::geometry::polygon<__int64,std::vector> & geom) Line 240  C++
    test.exe!mapnik::vector_tile_impl::transform_visitor<mapnik::vector_tile_impl::vector_tile_strategy,mapnik::vector_tile_impl::geometry_clipper<mapnik::vector_tile_impl::geometry_to_feature_pbf_visitor> >::operator()(const mapnik::geometry::polygon<double,mapnik::geometry::rings_container> & geom) Line 252  C++
    test.exe!mapbox::util::detail::dispatcher<mapnik::vector_tile_impl::transform_visitor<mapnik::vector_tile_impl::vector_tile_strategy,mapnik::vector_tile_impl::geometry_clipper<mapnik::vector_tile_impl::geometry_to_feature_pbf_visitor> > & __ptr64,mapnik::geometry::geometry<double>,void,mapnik::geometry::polygon<double,mapnik::geometry::rings_container>,mapnik::geometry::multi_point<double>,mapnik::geometry::multi_line_string<double>,mapnik::geometry::multi_polygon<double>,mapnik::geometry::geometry_collection<double> >::apply_const(const mapnik::geometry::geometry<double> & v, mapnik::vector_tile_impl::transform_visitor<mapnik::vector_tile_impl::vector_tile_strategy,mapnik::vector_tile_impl::geometry_clipper<mapnik::vector_tile_impl::geometry_to_feature_pbf_visitor> > & f) Line 313    C++
    test.exe!mapbox::util::detail::dispatcher<mapnik::vector_tile_impl::transform_visitor<mapnik::vector_tile_impl::vector_tile_strategy,mapnik::vector_tile_impl::geometry_clipper<mapnik::vector_tile_impl::geometry_to_feature_pbf_visitor> > & __ptr64,mapnik::geometry::geometry<double>,void,mapnik::geometry::line_string<double>,mapnik::geometry::polygon<double,mapnik::geometry::rings_container>,mapnik::geometry::multi_point<double>,mapnik::geometry::multi_line_string<double>,mapnik::geometry::multi_polygon<double>,mapnik::geometry::geometry_collection<double> >::apply_const(const mapnik::geometry::geometry<double> & v, mapnik::vector_tile_impl::transform_visitor<mapnik::vector_tile_impl::vector_tile_strategy,mapnik::vector_tile_impl::geometry_clipper<mapnik::vector_tile_impl::geometry_to_feature_pbf_visitor> > & f) Line 319  C++
    test.exe!mapbox::util::detail::dispatcher<mapnik::vector_tile_impl::transform_visitor<mapnik::vector_tile_impl::vector_tile_strategy,mapnik::vector_tile_impl::geometry_clipper<mapnik::vector_tile_impl::geometry_to_feature_pbf_visitor> > & __ptr64,mapnik::geometry::geometry<double>,void,mapnik::geometry::point<double>,mapnik::geometry::line_string<double>,mapnik::geometry::polygon<double,mapnik::geometry::rings_container>,mapnik::geometry::multi_point<double>,mapnik::geometry::multi_line_string<double>,mapnik::geometry::multi_polygon<double>,mapnik::geometry::geometry_collection<double> >::apply_const(const mapnik::geometry::geometry<double> & v, mapnik::vector_tile_impl::transform_visitor<mapnik::vector_tile_impl::vector_tile_strategy,mapnik::vector_tile_impl::geometry_clipper<mapnik::vector_tile_impl::geometry_to_feature_pbf_visitor> > & f) Line 319  C++
    test.exe!mapbox::util::detail::dispatcher<mapnik::vector_tile_impl::transform_visitor<mapnik::vector_tile_impl::vector_tile_strategy,mapnik::vector_tile_impl::geometry_clipper<mapnik::vector_tile_impl::geometry_to_feature_pbf_visitor> > & __ptr64,mapnik::geometry::geometry<double>,void,mapnik::geometry::geometry_empty,mapnik::geometry::point<double>,mapnik::geometry::line_string<double>,mapnik::geometry::polygon<double,mapnik::geometry::rings_container>,mapnik::geometry::multi_point<double>,mapnik::geometry::multi_line_string<double>,mapnik::geometry::multi_polygon<double>,mapnik::geometry::geometry_collection<double> >::apply_const(const mapnik::geometry::geometry<double> & v, mapnik::vector_tile_impl::transform_visitor<mapnik::vector_tile_impl::vector_tile_strategy,mapnik::vector_tile_impl::geometry_clipper<mapnik::vector_tile_impl::geometry_to_feature_pbf_visitor> > & f) Line 319 C++
    test.exe!mapbox::util::variant<mapnik::geometry::geometry_empty,mapnik::geometry::point<double>,mapnik::geometry::line_string<double>,mapnik::geometry::polygon<double,mapnik::geometry::rings_container>,mapnik::geometry::multi_point<double>,mapnik::geometry::multi_line_string<double>,mapnik::geometry::multi_polygon<double>,mapnik::geometry::geometry_collection<double> >::visit<mapnik::vector_tile_impl::transform_visitor<mapnik::vector_tile_impl::vector_tile_strategy,mapnik::vector_tile_impl::geometry_clipper<mapnik::vector_tile_impl::geometry_to_feature_pbf_visitor> > & __ptr64,mapnik::geometry::geometry<double>,void>(const mapnik::geometry::geometry<double> & v, mapnik::vector_tile_impl::transform_visitor<mapnik::vector_tile_impl::vector_tile_strategy,mapnik::vector_tile_impl::geometry_clipper<mapnik::vector_tile_impl::geometry_to_feature_pbf_visitor> > & f) Line 865 C++
    test.exe!mapnik::util::apply_visitor<mapnik::vector_tile_impl::transform_visitor<mapnik::vector_tile_impl::vector_tile_strategy,mapnik::vector_tile_impl::geometry_clipper<mapnik::vector_tile_impl::geometry_to_feature_pbf_visitor> > & __ptr64,mapnik::geometry::geometry<double> >(mapnik::vector_tile_impl::transform_visitor<mapnik::vector_tile_impl::vector_tile_strategy,mapnik::vector_tile_impl::geometry_clipper<mapnik::vector_tile_impl::geometry_to_feature_pbf_visitor> > & f, const mapnik::geometry::geometry<double> & v) Line 43    C++
    test.exe!mapnik::vector_tile_impl::detail::create_geom_layer(mapnik::vector_tile_impl::tile_layer & layer, double simplify_distance, double area_threshold, mapnik::vector_tile_impl::polygon_fill_type fill_type, bool strictly_simple, bool multi_polygon_union, bool process_all_rings) Line 167 C++
    test.exe!mapnik::vector_tile_impl::processor::update_tile(mapnik::vector_tile_impl::tile & t, double scale_denom, int offset_x, int offset_y) Line 316  C++
    test.exe!main(int argc, char * * argv) Line 25  C++
    [External Code] 

and pauses here in C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\include\vector:

Values of bnd in active_bound_list.hpp:164 before the exception occurs:

Also ((*bnd)->next_edge) seems to be null here:

And (*bnd)->edges having two almost identical edges:

@flippmoke anything else I should look for?

[DiabloRusso]

@BergWerkGIS you reproduce my bug! I use Visual Studio 2015 Update 2, if it has meaning. Thanx!

[wilhelmberg]

@flippmoke did you try with the provided shapefile? At first glance it looks like one of those that cause trouble when being cut into tiles.

Some analysis:

λ shapeindex --index-parts simplified_land_polygons.shp
max tree depth:8
split ratio:0.55
processing simplified_land_polygons.shp
9994
length=250574
version=1000
type=5
extent:box2d(-20037508.3427892476320267,-20037508.3688470497727394,20037508.3427892476320267,18461504.1001019775867462)
 number shapes=62648
 number nodes=2328
done!

EPSG:3857 "WGS 84 / Pseudo Mercator"

• single part features: 62631
• multi part features: 0
• null geometry features: 0
• all features: 62631

BBoxes:

I tried exporting to mbtiles via Mapbox Studio Classic and that worked. Checking geometry validity via QGIS gets stuck and crashes QGIS at some feature with fid>62.000 - still looking for the exact feature.

[wilhelmberg]

Removing the features that made QGIS hang didn't change the mapnik error.

fid 62056

fid 62261

fid 62620

[wilhelmberg]

Ah, I must have been blind, just realized the demo project has just Debug configuration.

@DiabloRusso did you build mapnik and all dependencies yourself, or did you use the mapnik-sdk we provide?

Our SDK is a Release build with debug symbols enabled, not a real Debug build. You cannot mix Release and Debug, that leads to strange unexpected behavior - like the one we are seeing.

If you used the mapnik-sdk please change your project to Release and try again.

[DiabloRusso]

@BergWerkGIS I modified mapbox/windows-builds to build a debug version of mapnik and all dependencies, it was not so difficult.

[flippmoke]

@BergWerkGIS Yes, I built a test in mapnik-vector-tile based on the provided test data and xml in the project. I never had any of the issues you are seeing.

Next edge should be one ahead of the edge there, so it is rather odd that it would be null unless that is the same as it being the end() in this case?

I see that I accidentally wasn't initializing it in the constructor -- which I am going to fix and push to master of wagyu, but it still should have been overwritten by this:

https://github.com/mapbox/wagyu/blob/master/include/mapbox/geometry/wagyu/local_minimum_util.hpp#L302

which is call right after the edge is created.

[wilhelmberg]

I modified mapbox/windows-builds to build a debug version of mapnik and all dependencies, it was not so difficult.

@DiabloRusso Oh Wow! The last time I tried there was no way getting everything to build Debug. VS constantly crashed, 32GB RAM were not enough (plus lots of swap space) and some intermediate files hit some kind of limit - if I remember correctly.

Would you mind creating a PR at mapbox/windows-builds?

[wilhelmberg]

Would you mind creating a PR at mapbox/windows-builds?

@DiabloRusso And I suppose changes to mapnik/mapnik-gyp were also necessary?

[wilhelmberg]

I used the script below on a PowerShell Dev prompt from the root of the test project like so .\linktype.ps1 .\

Looks like at least gdal and libtiff are Release. Is gdal involved with shape file reading?

For the ones prefixed with ??? I couldn't find a way to determine their configuration.

function check($filter){

    Write-Host "`nchecking $filter";

    foreach ($item in Get-ChildItem $args[0] -Recurse -Include $filter){
        $output = dumpbin /DIRECTIVES $item.FullName;
        if ($output -like '*LIBCMT*') { #MT and MTd
            Write-Host "MT or MTd $item";
        }elseIf ($output -like '*MSVCRTD*') { #MDd
            Write-Host "MDd $item";
        } else {
            $output = dumpbin /SYMBOLS $item.FullName;
            if($output -like '*COFF SYMBOL TABLE*'){ #debug
                Write-Host "debug $item";
            } else {
                $output = dumpbin /DEPENDENTS $item.FullName;
                if($output -like '*VCRUNTIME140D.dll*'){ #debug
                    Write-Host "debug $item";
                } elseif($output -like '*VCRUNTIME140.dll*'){ #release
                     Write-Host "release $item";
                } else {
                    Write-Host "??? $item";
                }
            }
        }
    }
}

check('*.lib');
check('*.dll');
check('*.exe');

Write-Host 'finished';

---

checking *.lib
debug C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\boost_python-vc140-mt-gd-1_62.lib
MDd C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\cairo-static.lib
debug C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\cairo.lib
MDd C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\freetype.lib
debug C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\gdal_i.lib
MDd C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\harfbuzz.lib
debug C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\icudt.lib
debug C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\icuind.lib
debug C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\icuucd.lib
debug C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\jpeg.lib
MDd C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\libboost_chrono-vc140-mt-gd-1_62.lib
MDd C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\libboost_date_time-vc140-mt-gd-1_62.lib
MDd C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\libboost_filesystem-vc140-mt-gd-1_62.lib
MDd C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\libboost_iostreams-vc140-mt-gd-1_62.lib
MDd C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\libboost_prg_exec_monitor-vc140-mt-gd-1_62.lib
MDd C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\libboost_program_options-vc140-mt-gd-1_62.lib
MDd C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\libboost_regex-vc140-mt-gd-1_62.lib
MDd C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\libboost_system-vc140-mt-gd-1_62.lib
MDd C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\libboost_test_exec_monitor-vc140-mt-gd-1_62.lib
MDd C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\libboost_thread-vc140-mt-gd-1_62.lib
MDd C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\libboost_timer-vc140-mt-gd-1_62.lib
MDd C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\libboost_unit_test_framework-vc140-mt-gd-1_62.lib
MDd C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\libboost_zlib-vc140-mt-gd-1_62.lib
debug C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\libexpat.lib
debug C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\libpng16.lib
MDd C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\libpqd.lib
MDd C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\libprotobuf-lite.lib
debug C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\libtiff_i.lib
debug C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\libwebp_debug_dll.lib
MDd C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\mapnik-json.lib
MDd C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\mapnik-wkt.lib
debug C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\mapnik.lib
debug C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\proj.lib
MDd C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\sqlite3.lib
MDd C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\zlib.lib
debug C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\zlibwapi.lib

checking *.dll
debug C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\boost_python-vc140-mt-gd-1_62.dll
debug C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\cairo.dll
release C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\gdal201.dll
??? C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\icudt56.dll
debug C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\icuin56d.dll
debug C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\icuuc56d.dll
debug C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\jpeg62.dll
??? C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\libexpat.dll
debug C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\libpng16.dll
debug C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\libpqd.dll
release C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\libtiff.dll
debug C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\libwebp_debug.dll
debug C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\mapnik.dll
debug C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\lib\zlibwapi.dll

checking *.exe
debug C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\bin\mapnik-index.exe
debug C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\bin\mapnik-render.exe
debug C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\bin\protoc.exe
debug C:\mb\_TEMP\mapnik-test\mapnik-test\lib\x64\Debug\mapnik-sdk\bin\shapeindex.exe
finished

[wilhelmberg]

@flippmoke paused before execution of line 164:

• there are 2 edges in (*bnd)->edges
• (*bnd)->current_edge points to the second one
• (*bnd)->next_edge is NULL
• trying to look into (*bnd)->edges.end() shows an error

[flippmoke]

Does current_edge point to the second one before or after the execution of line 162? If current_edge -- they both should be shifting together, it is fine if next_edge is equal to end() if the current_edge is equal to the last element in edges. Keep in mind that the pointer to end *end() should be null as it is not pointing to any element in edges. So if *next_edges is null it could be equal to end().

[DiabloRusso]

Would you mind creating a PR at mapbox/windows-builds?

@BergWerkGIS Ok, I will create a PR during 2 days!

The last time I tried there was no way getting everything to build Debug. VS constantly crashed, 32GB RAM were not enough (plus lots of swap space) and some intermediate files hit some kind of limit - if I remember correctly.

Is strangely, my first attempt to build mapnik I was doing on notebook with i3 processor, 4GB RAM and it took me nearly 8GB HDD

And I suppose changes to mapnik/mapnik-gyp were also necessary?

Now I can not say for sure, I don't remember that in this project I did changes. In most cases, I did changes in batch files. I will prepare my improved project and you will see all.

Thanks for testing my sample!

[wilhelmberg]

@DiabloRusso perfect!

@flippmoke video of stepping through those lines: mapnik-test.mp4.zip

[DiabloRusso]

@BergWerkGIS I create PR! Will need to make further changes in mapnik/mapnik-gyp, in build.bat exactly! Also I create issue in mapbox/windows-builds

[flippmoke]

I still don't see any way still that this bug is even possible in current code in wagyu, so my thought is we are having a memory corruption (likely upstream) from this code.

[DiabloRusso]

@flippmoke within an hour, I'll create PR into Wagyu, I think I found an error...

[flippmoke]

@DiabloRusso thanks! It is very hard to debug when I can't reproduce locally!

[DiabloRusso]

@flippmoke Before the first get next_edge, it is never initialized! Without going into essence of Wagyu, I found only one place where next_edge would may be initialized. I create PR at Wagyu, it solve my problem! Thanx!

[flippmoke]

@DiabloRusso thanks so much for hunting this down, its because next_edge is actually never used in the hot pixel portion of the code. The MSFT compiler is much more strict and noticed that we were iterating on a nullptr, while the operations would just more around unassigned memory in the gcc and clang configuration. I am rather sad that our static analysis tools didn't pick this up.

[DiabloRusso]

@flippmoke I just clean the bug, but has not check the quality of features in vector tiles. I notified you about my tests. Thanx! @BergWerkGIS thanx for test!

[springmeyer]

Amazing collaboration on this everyone 👏