search

mapbox / mapnik-vector-tile

Mapnik implemention of Mapbox Vector Tile specification
BSD 3-Clause "New" or "Revised" License
553 stars 117 forks source link

occasional test failure in v2_spec branch #173

Closed springmeyer closed 8 years ago

springmeyer commented 8 years ago

Looks like we've still got some memory corruption lurking in the v2_spec branch. I'm seeing this happen sometimes when running make test:

$ make test
make[1]: Nothing to be done for `all'.
./build/Release/tests

tests is a Catch v1.1 b3 host application.
Run with -? for options

-------------------------------------------------------------------------------
geometries visual tests
-------------------------------------------------------------------------------
../test/geometry_visual_test.cpp:265
...............................................................................

../test/geometry_visual_test.cpp:69: FAILED:
  CHECK( buffer2 == buffer )
with expansion:
  "I
?????(? x"      ??ZF?>??>?? ?   ??  ?"?!?
  ==
  "I
??(??er"5   ??ZF?>??>?? ?   ??  ?"?!?
                                                  x"

../test/utils/geometry_equal.hpp:140: FAILED:
  REQUIRE( p.template get<0>().y == Approx(p.template get<1>().y) )
with expansion:
  40.6390923571 == Approx( 40.6137897117 )

===============================================================================
test cases:    181 |    180 passed | 1 failed
assertions: 544800 | 544798 passed | 2 failed

make: *** [test] Error 2

/cc @flippmoke

springmeyer commented 8 years ago

geometry_clipper::geometry_clipper looks very suspect.

springmeyer commented 8 years ago

In these visitor callbacks std::move is being called on geom & (geometry reference) which seems like a possible way to corrupt memory: https://github.com/mapbox/mapnik-vector-tile/blob/v2_spec/src/vector_tile_geometry_clipper.ipp#L117-L139

springmeyer commented 8 years ago

std::move should only be called on temporaries.

springmeyer commented 8 years ago

Also built with latest clang++ to see if that could isolate any other problems, and sure enough:

../../src/vector_tile_processor.ipp:483:31: warning: moving a temporary object prevents copy elision [-Wpessimizing-move]
            lay_vec.push_back(std::move(std::async(
                              ^
../../src/vector_tile_processor.ipp:483:31: note: remove std::move call here
            lay_vec.push_back(std::move(std::async(
                              ^~~~~~~~~~
../../src/vector_tile_processor.ipp:504:31: warning: moving a temporary object prevents copy elision [-Wpessimizing-move]
            lay_vec.push_back(std::move(std::async(
                              ^
../../src/vector_tile_processor.ipp:504:31: note: remove std::move call here
            lay_vec.push_back(std::move(std::async(
                              ^~~~~~~~~~
2 warnings generated.
In file included from ../../src/vector_tile_geometry_clipper.cpp:2:
../../src/vector_tile_geometry_clipper.ipp:136:13: warning: moving a temporary object prevents copy elision [-Wpessimizing-move]
        g = std::move(mapnik::util::apply_visitor((*this), g));
            ^
../../src/vector_tile_geometry_clipper.ipp:136:13: note: remove std::move call here
        g = std::move(mapnik::util::apply_visitor((*this), g));
            ^~~~~~~~~~                                       ~
1 warning generated.
springmeyer commented 8 years ago

Warnings ^^ above are fixed now but the test failure still persists at 0a53bd8. Replicable with:

$ ./build/Release/tests "geometries visual tests"

tests is a Catch v1.1 b3 host application.
Run with -? for options

-------------------------------------------------------------------------------
geometries visual tests
-------------------------------------------------------------------------------
../../test/geometry_visual_test.cpp:265
...............................................................................

../../test/geometry_visual_test.cpp:69: FAILED:
  CHECK( buffer2 == buffer )
with expansion:
  "H
  layer"4   ?*????  ????
                            ?
                             ???
                                ??
  ?f???;(? x"
  ==
  "H
  layer"4   ?*????  ????
                            ?
                             ???
                                ??
  ?f???;(? "

===============================================================================
test cases:      1 |      0 passed | 1 failed
assertions: 658340 | 658339 passed | 1 failed
springmeyer commented 8 years ago

Update:

1) @flippmoke thoughr this patch would avoid the occasional failures:

diff --git a/include/mapnik/json/positions_grammar.hpp b/include/mapnik/json/positions_grammar.hpp
index 1535422..f4a96db 100644
--- a/include/mapnik/json/positions_grammar.hpp
+++ b/include/mapnik/json/positions_grammar.hpp
@@ -56,7 +56,11 @@ struct set_position_impl
     template <typename T0,typename T1>
     result_type operator() (T0 & coords, T1 const& pos) const
     {
-        if (pos) coords = *pos;
+        if (pos)
+         {
+             auto const& p = *pos;
+             coords = p;
+         }
     }
 };

@@ -66,7 +70,12 @@ struct push_position_impl
     template <typename T0, typename T1>
     result_type operator() (T0 & coords, T1 const& pos) const
     {
-        if (pos) coords.emplace_back(*pos);
+        if (pos)
+        {
+            auto const& p = *pos;
+            typename T0::value_type p1(p);
+            coords.emplace_back(p1);
+        }
     }
 };

For me I am still able to replicate locally with that patch.

2) Crash backtrace

Running multiple tests at once:

./build/Release/tests "geometries visual tests" & ./build/Release/tests "geometries visual tests" & ./build/Release/tests "geometries visual tests" & ./build/Release/tests "geometries visual tests" & ./build/Release/tests "geometries visual tests"

I'm able to trigger this crash:

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libsystem_malloc.dylib          0x00007fff910e6761 tiny_malloc_from_free_list + 149
1   libsystem_malloc.dylib          0x00007fff910e5715 szone_malloc_should_clear + 292
2   libsystem_malloc.dylib          0x00007fff910e55b1 malloc_zone_malloc + 71
3   libsystem_malloc.dylib          0x00007fff910e40dc malloc + 42
4   libc++.1.dylib                  0x00007fff8e8bf7de operator new(unsigned long) + 30
5   libc++.1.dylib                  0x00007fff8e8c2205 std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::__init(char const*, unsigned long) + 59
6   tests                           0x000000010d8dca07 Catch::RunContext::assertionEnded(Catch::AssertionResult const&) + 343
7   tests                           0x000000010d8c1c34 Catch::ResultBuilder::captureExpression() + 164
8   tests                           0x000000010d82c2db void geometry_equal_visitor::operator()<double>(mapnik::geometry::line_string<double> const&, mapnik::geometry::line_string<double> const&) + 1115
9   tests                           0x000000010d83169d void geometry_equal_visitor::operator()<double>(mapnik::geometry::polygon<double, mapnik::geometry::rings_container> const&, mapnik::geometry::polygon<double, mapnik::geometry::rings_container> const&) + 45
10  tests                           0x000000010d81a2b9 void assert_g_equal<double>(mapnik::geometry::geometry<double> const&, mapnik::geometry::geometry<double> const&) + 969
11  tests                           0x000000010d818420 clip_geometry(mapnik::Map const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, mapnik::box2d<double> const&, int, bool, mapnik::vector_tile_impl::polygon_fill_type, bool, bool) + 12736
12  tests                           0x000000010d81991a ____C_A_T_C_H____T_E_S_T____265() + 3466
13  tests                           0x000000010d8dfdea Catch::RunContext::runCurrentTest(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&) + 810
14  tests                           0x000000010d8dbd7e Catch::RunContext::runTest(Catch::TestCase const&) + 590
15  tests                           0x000000010d8d5f4c Catch::Runner::runTests() + 1628
16  tests                           0x000000010d8d182c Catch::Session::run() + 540
17  tests                           0x000000010d8c3db6 main + 166
18  libdyld.dylib                   0x00007fff8f4615ad start + 1
springmeyer commented 8 years ago

Also replicates when mapnik-vt is built in Debug mode:

$ ./build/Debug/tests "geometries visual tests" & ./build/Debug/tests "geometries visual tests" & ./build/Debug/tests "geometries visual tests" & ./build/Debug/tests "geometries visual tests" & ./build/Debug/tests "geometries visual tests" & ./build/Debug/tests "geometries visual tests"
[3] 77971
[4] 77972
[5] 77973
[6] 77974
[7] 77975

tests is a Catch v1.1 b3 host application.
Run with -? for options

-------------------------------------------------------------------------------
geometries visual tests
-------------------------------------------------------------------------------
../../test/geometry_visual_test.cpp:265
...............................................................................

../../test/geometry_visual_test.cpp:179: FAILED:
  CHECK( expected_string == geojson_string )
with expansion:
  "{"tyf": "Point", "coordinates":[], "properties":{"message":"Tile had no
  layers"}}"
  ==
  "{"type": "Point", "coordinates":[], "properties":{"message":"Tile had no
  layers"}}"
springmeyer commented 8 years ago

One twist: this is impacting the strings used to find testcases such that it looks like the testdata may get modified unintentionally:

~/projects/mapnik-vector-tile/test/geometry-test-data[(HEAD detached at 394e4de)]$ git st
## HEAD (no branch)
?? "output/coun\001fr-clockwise-polygon-counter-clockwise-hole/"
springmeyer commented 8 years ago

New backtrace:

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   tests                           0x000000010592e84c boost::geometry::traits::access<mapnik::geometry::point<double>, 1ul, void>::get(mapnik::geometry::point<double> const&) + 12 (geometry_adapters.hpp:48)
1   tests                           0x000000010592e835 boost::geometry::core_dispatch::access<boost::geometry::point_tag, mapnik::geometry::point<double>, double, 1ul, boost::integral_constant<bool, false> >::get(mapnik::geometry::point<double> const&) + 21 (access.hpp:169)
2   tests                           0x000000010592e695 boost::geometry::coordinate_type<mapnik::geometry::point<double> >::type boost::geometry::get<1ul, mapnik::geometry::point<double> >(mapnik::geometry::point<double> const&, boost::geometry::detail::signature_getset_dimension*) + 37 (access.hpp:285)
3   tests                           0x000000010598d4b1 int boost::geometry::strategy::within::winding<mapnik::geometry::point<double>, mapnik::geometry::point<double>, void>::check_segment<1ul>(mapnik::geometry::point<double> const&, mapnik::geometry::point<double> const&, mapnik::geometry::point<double> const&, boost::geometry::strategy::within::winding<mapnik::geometry::point<double>, mapnik::geometry::point<double>, void>::counter&, bool&, bool&) + 49 (point_in_poly_winding.hpp:266)
4   tests                           0x000000010598d154 boost::geometry::strategy::within::winding<mapnik::geometry::point<double>, mapnik::geometry::point<double>, void>::apply(mapnik::geometry::point<double> const&, mapnik::geometry::point<double> const&, mapnik::geometry::point<double> const&, boost::geometry::strategy::within::winding<mapnik::geometry::point<double>, mapnik::geometry::point<double>, void>::counter&) + 68 (point_in_poly_winding.hpp:307)
5   tests                           0x000000010598cfe9 int boost::geometry::detail::within::point_in_range<mapnik::geometry::point<double>, boost::geometry::detail::normalized_view<mapnik::geometry::linear_ring<double> const>, boost::geometry::strategy::within::winding<mapnik::geometry::point<double>, mapnik::geometry::point<double>, void> >(mapnik::geometry::point<double> const&, boost::geometry::detail::normalized_view<mapnik::geometry::linear_ring<double> const> const&, boost::geometry::strategy::within::winding<mapnik::geometry::point<double>, mapnik::geometry::point<double>, void> const&) + 169 (point_in_geometry.hpp:74)
6   tests                           0x000000010598cf03 int boost::geometry::detail_dispatch::within::point_in_geometry<mapnik::geometry::linear_ring<double>, boost::geometry::ring_tag>::apply<mapnik::geometry::point<double>, boost::geometry::strategy::within::winding<mapnik::geometry::point<double>, mapnik::geometry::point<double>, void> >(mapnik::geometry::point<double> const&, mapnik::geometry::linear_ring<double> const&, boost::geometry::strategy::within::winding<mapnik::geometry::point<double>, mapnik::geometry::point<double>, void> const&) + 83 (point_in_geometry.hpp:239)
7   tests                           0x000000010598ceaa int boost::geometry::detail::within::point_in_geometry<mapnik::geometry::point<double>, mapnik::geometry::linear_ring<double>, boost::geometry::strategy::within::winding<mapnik::geometry::point<double>, mapnik::geometry::point<double>, void> >(mapnik::geometry::point<double> const&, mapnik::geometry::linear_ring<double> const&, boost::geometry::strategy::within::winding<mapnik::geometry::point<double>, mapnik::geometry::point<double>, void> const&) + 42 (point_in_geometry.hpp:411)
8   tests                           0x000000010598ce55 bool boost::geometry::detail::covered_by::use_point_in_geometry::apply<mapnik::geometry::point<double>, mapnik::geometry::linear_ring<double>, boost::geometry::strategy::within::winding<mapnik::geometry::point<double>, mapnik::geometry::point<double>, void> >(mapnik::geometry::point<double> const&, mapnik::geometry::linear_ring<double> const&, boost::geometry::strategy::within::winding<mapnik::geometry::point<double>, mapnik::geometry::point<double>, void> const&) + 37 (covered_by.hpp:47)
9   tests                           0x000000010598cde9 bool boost::geometry::resolve_strategy::covered_by::apply<mapnik::geometry::point<double>, mapnik::geometry::linear_ring<double>, boost::geometry::strategy::within::winding<mapnik::geometry::point<double>, mapnik::geometry::point<double>, void> >(mapnik::geometry::point<double> const&, mapnik::geometry::linear_ring<double> const&, boost::geometry::strategy::within::winding<mapnik::geometry::point<double>, mapnik::geometry::point<double>, void> const&) + 57 (covered_by.hpp:274)
10  tests                           0x000000010598cda1 bool boost::geometry::resolve_strategy::covered_by::apply<mapnik::geometry::point<double>, mapnik::geometry::linear_ring<double> >(mapnik::geometry::point<double> const&, mapnik::geometry::linear_ring<double> const&, boost::geometry::default_strategy) + 33 (covered_by.hpp:305)
11  tests                           0x000000010598cd71 bool boost::geometry::resolve_variant::covered_by<mapnik::geometry::point<double>, mapnik::geometry::linear_ring<double> >::apply<boost::geometry::default_strategy>(mapnik::geometry::point<double> const&, mapnik::geometry::linear_ring<double> const&, boost::geometry::default_strategy const&) + 33 (covered_by.hpp:322)
12  tests                           0x000000010598b0a1 bool boost::geometry::covered_by<mapnik::geometry::point<double>, mapnik::geometry::linear_ring<double> >(mapnik::geometry::point<double> const&, mapnik::geometry::linear_ring<double> const&) + 33 (covered_by.hpp:443)
13  tests                           0x00000001059b7bd1 bool boost::geometry::detail::is_valid::is_valid_polygon<mapnik::geometry::polygon<double, mapnik::geometry::rings_container>, true>::are_holes_inside<std::__1::__wrap_iter<mapnik::geometry::linear_ring<double> const*>, mapnik::geometry::linear_ring<double>, boost::iterators::filter_iterator<boost::geometry::detail::is_valid::is_valid_multipolygon<mapnik::geometry::multi_polygon<double>, true>::has_multi_index, std::__1::__deque_iterator<boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >, boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >*, boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >&, boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >**, long, 26l> >, boost::geometry::failing_reason_policy<true, true> >(std::__1::__wrap_iter<mapnik::geometry::linear_ring<double> const*>, std::__1::__wrap_iter<mapnik::geometry::linear_ring<double> const*>, mapnik::geometry::linear_ring<double> const&, boost::iterators::filter_iterator<boost::geometry::detail::is_valid::is_valid_multipolygon<mapnik::geometry::multi_polygon<double>, true>::has_multi_index, std::__1::__deque_iterator<boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >, boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >*, boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >&, boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >**, long, 26l> >, boost::iterators::filter_iterator<boost::geometry::detail::is_valid::is_valid_multipolygon<mapnik::geometry::multi_polygon<double>, true>::has_multi_index, std::__1::__deque_iterator<boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >, boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >*, boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >&, boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >**, long, 26l> >, boost::geometry::failing_reason_policy<true, true>&) + 1953 (polygon.hpp:215)
14  tests                           0x00000001059b741f bool boost::geometry::detail::is_valid::is_valid_polygon<mapnik::geometry::polygon<double, mapnik::geometry::rings_container>, true>::are_holes_inside<std::__1::vector<mapnik::geometry::linear_ring<double>, std::__1::allocator<mapnik::geometry::linear_ring<double> > >, mapnik::geometry::linear_ring<double>, boost::iterators::filter_iterator<boost::geometry::detail::is_valid::is_valid_multipolygon<mapnik::geometry::multi_polygon<double>, true>::has_multi_index, std::__1::__deque_iterator<boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >, boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >*, boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >&, boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >**, long, 26l> >, boost::geometry::failing_reason_policy<true, true> >(std::__1::vector<mapnik::geometry::linear_ring<double>, std::__1::allocator<mapnik::geometry::linear_ring<double> > > const&, mapnik::geometry::linear_ring<double> const&, boost::iterators::filter_iterator<boost::geometry::detail::is_valid::is_valid_multipolygon<mapnik::geometry::multi_polygon<double>, true>::has_multi_index, std::__1::__deque_iterator<boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >, boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >*, boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >&, boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >**, long, 26l> >, boost::iterators::filter_iterator<boost::geometry::detail::is_valid::is_valid_multipolygon<mapnik::geometry::multi_polygon<double>, true>::has_multi_index, std::__1::__deque_iterator<boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >, boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >*, boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >&, boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >**, long, 26l> >, boost::geometry::failing_reason_policy<true, true>&) + 287 (polygon.hpp:274)
15  tests                           0x00000001059b704d bool boost::geometry::detail::is_valid::is_valid_polygon<mapnik::geometry::polygon<double, mapnik::geometry::rings_container>, true>::has_holes_inside::apply<boost::iterators::filter_iterator<boost::geometry::detail::is_valid::is_valid_multipolygon<mapnik::geometry::multi_polygon<double>, true>::has_multi_index, std::__1::__deque_iterator<boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >, boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >*, boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >&, boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >**, long, 26l> >, boost::geometry::failing_reason_policy<true, true> >(mapnik::geometry::polygon<double, mapnik::geometry::rings_container> const&, boost::iterators::filter_iterator<boost::geometry::detail::is_valid::is_valid_multipolygon<mapnik::geometry::multi_polygon<double>, true>::has_multi_index, std::__1::__deque_iterator<boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >, boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >*, boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >&, boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >**, long, 26l> >, boost::iterators::filter_iterator<boost::geometry::detail::is_valid::is_valid_multipolygon<mapnik::geometry::multi_polygon<double>, true>::has_multi_index, std::__1::__deque_iterator<boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >, boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >*, boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >&, boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >**, long, 26l> >, boost::geometry::failing_reason_policy<true, true>&) + 237 (polygon.hpp:290)
16  tests                           0x00000001059b6e71 bool boost::geometry::detail::is_valid::is_valid_multipolygon<mapnik::geometry::multi_polygon<double>, true>::has_property_per_polygon<boost::geometry::detail::is_valid::is_valid_polygon<mapnik::geometry::polygon<double, mapnik::geometry::rings_container>, true>::has_holes_inside>::apply<std::__1::__wrap_iter<mapnik::geometry::polygon<double, mapnik::geometry::rings_container> const*>, std::__1::__deque_iterator<boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >, boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >*, boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >&, boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >**, long, 26l>, boost::geometry::failing_reason_policy<true, true> >(std::__1::__wrap_iter<mapnik::geometry::polygon<double, mapnik::geometry::rings_container> const*>, std::__1::__wrap_iter<mapnik::geometry::polygon<double, mapnik::geometry::rings_container> const*>, std::__1::__deque_iterator<boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >, boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >*, boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >&, boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >**, long, 26l>, std::__1::__deque_iterator<boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >, boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >*, boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >&, boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >**, long, 26l>, boost::geometry::failing_reason_policy<true, true>&) + 641 (multipolygon.hpp:183)
17  tests                           0x00000001059b117d bool boost::geometry::detail::is_valid::is_valid_multipolygon<mapnik::geometry::multi_polygon<double>, true>::have_holes_inside<std::__1::__wrap_iter<mapnik::geometry::polygon<double, mapnik::geometry::rings_container> const*>, std::__1::__deque_iterator<boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >, boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >*, boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >&, boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >**, long, 26l>, boost::geometry::failing_reason_policy<true, true> >(std::__1::__wrap_iter<mapnik::geometry::polygon<double, mapnik::geometry::rings_container> const*>, std::__1::__wrap_iter<mapnik::geometry::polygon<double, mapnik::geometry::rings_container> const*>, std::__1::__deque_iterator<boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >, boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >*, boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >&, boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >**, long, 26l>, std::__1::__deque_iterator<boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >, boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >*, boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >&, boost::geometry::detail::overlay::turn_info<mapnik::geometry::point<double>, boost::geometry::segment_ratio<long long>, boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, boost::array<boost::geometry::detail::overlay::turn_operation<boost::geometry::segment_ratio<long long> >, 2ul> >**, long, 26l>, boost::geometry::failing_reason_policy<true, true>&) + 125 (multipolygon.hpp:209)
18  tests                           0x00000001059b0b9e bool boost::geometry::detail::is_valid::is_valid_multipolygon<mapnik::geometry::multi_polygon<double>, true>::apply<boost::geometry::failing_reason_policy<true, true> >(mapnik::geometry::multi_polygon<double> const&, boost::geometry::failing_reason_policy<true, true>&) + 1102 (multipolygon.hpp:299)
19  tests                           0x00000001059b0742 bool boost::geometry::resolve_variant::is_valid<mapnik::geometry::multi_polygon<double> >::apply<boost::geometry::failing_reason_policy<true, true> >(mapnik::geometry::multi_polygon<double> const&, boost::geometry::failing_reason_policy<true, true>&) + 34 (interface.hpp:41)
20  tests                           0x00000001059b070d bool boost::geometry::is_valid<mapnik::geometry::multi_polygon<double>, boost::geometry::failing_reason_policy<true, true> >(mapnik::geometry::multi_polygon<double> const&, boost::geometry::failing_reason_policy<true, true>&) + 29 (interface.hpp:79)
21  tests                           0x00000001059b02f5 bool boost::geometry::is_valid<mapnik::geometry::multi_polygon<double> >(mapnik::geometry::multi_polygon<double> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&) + 1173 (interface.hpp:151)
22  tests                           0x00000001059afe50 bool mapnik::geometry::detail::geometry_is_valid_string::operator()<double>(mapnik::geometry::multi_polygon<double> const&) const + 32 (geometry_is_valid.hpp:234)
23  tests                           0x0000000105954187 bool mapnik::geometry::is_valid<double>(mapnik::geometry::geometry<double> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&) + 2055 (variant.hpp:297)
24  tests                           0x000000010594c19d clip_geometry(mapnik::Map const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, mapnik::box2d<double> const&, int, bool, mapnik::vector_tile_impl::polygon_fill_type, bool, bool) + 3789 (geometry_visual_test.cpp:118)
25  tests                           0x0000000105951fb8 ____C_A_T_C_H____T_E_S_T____265() + 9528 (geometry_visual_test.cpp:302)
26  tests                           0x0000000105a5ad63 Catch::FreeFunctionTestCase::invoke() const + 19 (catch.hpp:5827)
27  tests                           0x0000000105a3d4ad Catch::TestCase::invoke() const + 45 (catch.hpp:6732)
28  tests                           0x0000000105aba0f9 Catch::RunContext::invokeActiveTestCase() + 25 (catch.hpp:5425)
29  tests                           0x0000000105ab62f9 Catch::RunContext::runCurrentTest(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&) + 1225 (catch.hpp:5397)
30  tests                           0x0000000105aa88e3 Catch::RunContext::runTest(Catch::TestCase const&) + 643 (catch.hpp:5236)
31  tests                           0x0000000105a97201 Catch::Runner::runTests() + 2625 (catch.hpp:5555)
32  tests                           0x0000000105a83dd5 Catch::Session::run() + 437 (catch.hpp:5686)
33  tests                           0x0000000105a56c23 Catch::Session::run(int, char* const*) + 83 (catch.hpp:5666)
34  tests                           0x0000000105a51a5e main + 430 (test_main.cpp:19)
35  libdyld.dylib                   0x00007fff8f4615ad start + 1
springmeyer commented 8 years ago

Another (different):

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libsystem_malloc.dylib          0x00007fff910e6813 tiny_malloc_from_free_list + 327
1   libsystem_malloc.dylib          0x00007fff910e5715 szone_malloc_should_clear + 292
2   libsystem_malloc.dylib          0x00007fff910e55b1 malloc_zone_malloc + 71
3   libsystem_malloc.dylib          0x00007fff910e40dc malloc + 42
4   libc++.1.dylib                  0x00007fff8e8bf7de operator new(unsigned long) + 30
5   tests                           0x00000001073298f4 google::protobuf::internal::GenericTypeHandler<vector_tile::Tile_Layer>::New() + 20 (repeated_field.h:362)
6   tests                           0x0000000107329896 google::protobuf::RepeatedPtrField<vector_tile::Tile_Layer>::TypeHandler::Type* google::protobuf::internal::RepeatedPtrFieldBase::Add<google::protobuf::RepeatedPtrField<vector_tile::Tile_Layer>::TypeHandler>() + 118 (repeated_field.h:905)
7   tests                           0x0000000107329815 google::protobuf::RepeatedPtrField<vector_tile::Tile_Layer>::Add() + 21 (repeated_field.h:1124)
8   tests                           0x000000010732927c vector_tile::Tile::add_layers() + 28 (vector_tile.pb.h:1311)
9   tests                           0x0000000107694b9d vector_tile::Tile::MergePartialFromCodedStream(google::protobuf::io::CodedInputStream*) + 1405 (vector_tile.pb.cc:1354)
10  tests                           0x000000010771e865 google::protobuf::MessageLite::ParseFromString(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) + 661
11  tests                           0x000000010728d3f1 mapnik::vector_tile_impl::tile::get_tile() const + 49 (vector_tile_tile.hpp:138)
12  tests                           0x000000010728550b clip_geometry(mapnik::Map const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, mapnik::box2d<double> const&, int, bool, mapnik::vector_tile_impl::polygon_fill_type, bool, bool) + 571 (geometry_visual_test.cpp:62)
13  tests                           0x000000010728bc5e ____C_A_T_C_H____T_E_S_T____265() + 8670 (geometry_visual_test.cpp:296)
14  tests                           0x0000000107394d63 Catch::FreeFunctionTestCase::invoke() const + 19 (catch.hpp:5827)
15  tests                           0x00000001073774ad Catch::TestCase::invoke() const + 45 (catch.hpp:6732)
16  tests                           0x00000001073f40f9 Catch::RunContext::invokeActiveTestCase() + 25 (catch.hpp:5425)
17  tests                           0x00000001073f02f9 Catch::RunContext::runCurrentTest(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&) + 1225 (catch.hpp:5397)
18  tests                           0x00000001073e28e3 Catch::RunContext::runTest(Catch::TestCase const&) + 643 (catch.hpp:5236)
19  tests                           0x00000001073d1201 Catch::Runner::runTests() + 2625 (catch.hpp:5555)
20  tests                           0x00000001073bddd5 Catch::Session::run() + 437 (catch.hpp:5686)
21  tests                           0x0000000107390c23 Catch::Session::run(int, char* const*) + 83 (catch.hpp:5666)
22  tests                           0x000000010738ba5e main + 430 (test_main.cpp:19)
23  libdyld.dylib                   0x00007fff8f4615ad start + 1
springmeyer commented 8 years ago

This patch works around the issue for me locally (can no longer replicate test failure):

diff --git a/src/vector_tile_processor.ipp b/src/vector_tile_processor.ipp
index d7ff80d..16d721b 100644
--- a/src/vector_tile_processor.ipp
+++ b/src/vector_tile_processor.ipp
@@ -477,7 +477,7 @@ MAPNIK_VECTOR_INLINE void processor::update_tile(tile & t,
         if (ds->type() == datasource::Vector)
         {
             lay_vec.push_back(std::async(
-                        //std::launch::deferred, // uncomment to make single threaded  
+                        std::launch::deferred, // uncomment to make single threaded  
                         detail::create_geom_layer,
                         ds,
                         q,
@@ -498,7 +498,7 @@ MAPNIK_VECTOR_INLINE void processor::update_tile(tile & t,
         else // Raster
         {
             lay_vec.push_back(std::async(
-                        //std::launch::deferred, // uncomment to make single threaded  
+                        std::launch::deferred, // uncomment to make single threaded  
                         detail::create_raster_layer,
                         ds,
                         q,
flippmoke commented 8 years ago

I remember earlier why I had a std::move on that, it was because std future as far as I have heard are moveable, but not copyable. I wonder if we are running into problems due to the way we putting a std future into a std vector?

springmeyer commented 8 years ago

@flippmoke my hunch is that something is fundamentally not thread safe about the objects being worked with such that std::async is leading to corrupted memory.

flippmoke commented 8 years ago

I was curious if it was related to the references being used in the methods because they might go out of scope, I know that if a reference is used internally std async should make a copy, but to confirm I did the following change but the errors still occur.

diff --git a/src/vector_tile_processor.ipp b/src/vector_tile_processor.ipp
index d7ff80d..1a11f39 100644
--- a/src/vector_tile_processor.ipp
+++ b/src/vector_tile_processor.ipp
@@ -189,13 +189,13 @@ inline void create_geom_feature(T const& vs,
 }

 inline tile_layer create_geom_layer(mapnik::datasource_ptr ds,
-                                    mapnik::query const& q,
-                                    std::string const& layer_name,
+                                    mapnik::query q,
+                                    std::string layer_name,
                                     std::uint32_t layer_extent,
-                                    std::string const& target_proj_srs,
-                                    std::string const& source_proj_srs,
-                                    mapnik::view_transform const& view_trans,
-                                    mapnik::box2d<double> const& buffered_extent,
+                                    std::string target_proj_srs,
+                                    std::string source_proj_srs,
+                                    mapnik::view_transform view_trans,
+                                    mapnik::box2d<double> buffered_extent,
                                     double simplify_distance,
                                     double area_threshold,
                                     polygon_fill_type fill_type,
@@ -311,13 +311,13 @@ inline tile_layer create_geom_layer(mapnik::datasource_ptr ds,
 }

 inline tile_layer create_raster_layer(mapnik::datasource_ptr ds,
-                                      mapnik::query const& q,
-                                      std::string const& layer_name,
+                                      mapnik::query q,
+                                      std::string layer_name,
                                       std::uint32_t layer_extent,
-                                      std::string const& target_proj_srs,
-                                      std::string const& source_proj_srs,
-                                      mapnik::view_transform const& view_trans,
-                                      std::string const& image_format,
+                                      std::string target_proj_srs,
+                                      std::string source_proj_srs,
+                                      mapnik::view_transform view_trans,
+                                      std::string image_format,
                                       scaling_method_e scaling_method)
 {
     // Setup projection
flippmoke commented 8 years ago

Changing from the geojson to the memory datasource did not seem to prevent the problem:

diff --git a/test/geometry_visual_test.cpp b/test/geometry_visual_test.cpp
index 6ea8774..80a65a9 100644
--- a/test/geometry_visual_test.cpp
+++ b/test/geometry_visual_test.cpp
@@ -272,7 +272,7 @@ TEST_CASE("geometries visual tests")
     }
     for (std::string const& file: geometries)
     {
-        mapnik::datasource_ptr ds = testing::build_geojson_fs_ds(file);
+        mapnik::datasource_ptr ds = testing::build_geojson_ds(file);
         mapnik::box2d<double> bbox = ds->envelope();
         mapnik::Map map(256, 256, "+init=epsg:4326");
         mapnik::layer lyr("layer","+init=epsg:4326");
springmeyer commented 8 years ago

This boiled down to a bug in apple's libc++ which @flippmoke uncovered as https://bytecoin.org/blog/pthread-bug-osx/?utm_source=email7&amp;utm_medium=email&amp;u. For the time being therefore we will default to std::deferred and make it runtime toggleable so it can be tested on linux (or other implementations that are not buggy): https://github.com/mapbox/mapnik-vector-tile/commit/db25964db2b2d7dcde26a68832159320b6330f74