search

mapbox / tippecanoe

Build vector tilesets from large collections of GeoJSON features.
BSD 2-Clause "Simplified" License
2.72k stars 432 forks source link

Segmentation fault #840

Open stevage opened 4 years ago

stevage commented 4 years ago

I have been processing a file which is 81GB of newline-delimited JSON. I have processed it a couple of times successfully on this machine. Just now I changed the parameters slightly:

tippecanoe --output=mbtiles/parcels.mbtiles --force --read-parallel --minimum-zoom=13 --maximum-zoom=15 --generate-ids --simplification=10 --detect-shared-borders --simplify-only-low-zooms --attribute-type='ATTOM ID:int' --include='ATTOM ID' --include=APN --include=CountyFIPS --coalesce-smallest-as-needed --base-zoom=14 --accumulate-attribute=APN:comma --accumulate-attribute='ATTOM ID:comma' --layer=parcels data/parcels.json

and got a seg fault:

...
Going to try keeping the biggest 19.34% of the features to make it fit
tile 13/2352/3121 size is 502684 with detail 12, >500000
Going to try keeping the biggest 17.31% of the features to make it fit
Segmentation fault (core dumped)

Now, following this I'll try running it again to actually get the core dump, which seems to not be written by default.

The changes between "worked" and "crashed" are:

// worked:
                    minimumZoom: 13,
                    maximumZoom: 15, 
                    generateIds: true,
                    simplification:10,
                    filename: `${options.datapath}/parcels${options.suffix}.json`,
                    detectSharedBorders: true,
                    dropSmallestAsNeeded: true,
                    simplifyOnlyLowZooms: true,
                    attributeType: 'ATTOM ID:int',
                    include: ['ATTOM ID', 'APN','CountyFIPS'],

// crashed (all the above, plus these:)
                    dropSmallestAsNeeded: false,
                    coalesceSmallestAsNeeded: true,
                    baseZoom: 14,
                    accumulateAttribute: ['APN:comma', 'ATTOM ID:comma'],

Somewhere in the log I also noticed there was one tile that tippecanoe couldn't make small enough. What does it actually do in that situation? Previously I thought I had seen it abort altogether.

I have also managed to process a subset of this dataset successfully, with these exact settings.

I'm not sure how to share the source file with you, given its size. Any ideas?

tippecanoe v1.32.12

stevage commented 4 years ago

Oh, I see I'm a couple of versions behind. Trying again with 1.34.6

stevage commented 4 years ago

Slightly different output but still crashed:

Going to try keeping the biggest 17.31% of the features to make it fit
Segmentation fault080
springmeyer commented 4 years ago

@stevage thanks for reporting and helping narrow this down. Getting a backtrace out of the core dump would be ideal, since it should show where the crash is happening. That way we might be able to fix without replicating.

https://jvns.ca/blog/2018/04/28/debugging-a-segfault-on-linux/ looks good, but skip the valgrind step since that will slow down the program so much it might not be tenable. Also, you could use https://github.com/mapbox/logbt which does all the things in that blog post automatically.

stevage commented 4 years ago

Ah, great. Ok, I'm trying the logbt option.

$ ~/logbt/logbt -- ./4-make-tiles.js parcels
[logbt] using corefile location: /tmp/logbt-coredumps
[logbt] using core_pattern: core.%p.%E
4: Generating vector tiles
tippecanoe --output=mbtiles/parcels.mbtiles --force --read-parallel --minimum-zoom=13 --maximum-zoom=15 --generate-ids --simplification=10 --simplify-only-low-zooms --attribute-type='ATTOM ID:int' --include='ATTOM ID' --include=APN --include=CountyFIPS --coalesce-smallest-as-needed --base-zoom=14 --accumulate-attribute=APN:comma --accumulate-attribute='ATTOM ID:comma' --layer=parcels data/parcels.json
Read 1.64 million features
stevage commented 4 years ago

Ok, I have emailed you the dump.

stevage commented 4 years ago

Ok, finally got there.

tile 13/2352/3121 size is 506338 with detail 12, >500000
Going to try keeping the biggest 20.25% of the features to make it fit
Segmentation fault (core dumped)
Tiles for parcels:parcels failed in 4.02 h.
[logbt] saw './4-make-tiles.js' exit with code:139 (SEGV)
[logbt] Found corefile (non-tracked) at /tmp/logbt-coredumps/core.28612.!home!steve_bennett_cherre_com!tippecanoe!tippecanoe
[logbt] Processing cores...
[New LWP 30669]
[New LWP 30671]
[New LWP 30670]
[New LWP 30675]
[New LWP 30672]
[New LWP 30676]
[New LWP 30673]
[New LWP 30674]
[New LWP 28612]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `tippecanoe --output=mbtiles/parcels.mbtiles --force --read-parallel --minimum-z'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  deserialize_ulong_long (zigzag=<synthetic pointer>, f=<synthetic pointer>) at serial.cpp:97
97                      if ((**f & 0x80) == 0) {
[Current thread is 1 (Thread 0x7fc3a6d8a700 (LWP 30669))]

Thread 9 (Thread 0x7fc479cdbb80 (LWP 28612)):
#0  0x00007fc479386d2d in __GI___pthread_timedjoin_ex (threadid=140478294566656, thread_return=thread_return@entry=0x7ffe69de6bd8, abstime=abstime@entry=0x0, block=block@entry=true) at pthread_join_common.c:89
#1  0x00007fc479386b5c in __pthread_join (threadid=<optimized out>, thread_return=thread_return@entry=0x7ffe69de6bd8) at pthread_join.c:24
#2  0x00005645c7c7a394 in traverse_zooms (geomfd=<optimized out>, geom_size=<optimized out>, metabase=<optimized out>, stringpool=<optimized out>, midx=<optimized out>, midy=<optimized out>, maxzoom=<optimized out>, minzoom=<optimized out>, outdb=<optimized out>, outdir=<optimized out>, buffer=<optimized out>, fname=<optimized out>, tmpdir=<optimized out>, gamma=<optimized out>, full_detail=<optimized out>, low_detail=<optimized out>, min_detail=<optimized out>, meta_off=<optimized out>, pool_off=<optimized out>, initial_x=<optimized out>, initial_y=<optimized out>, simplification=<optimized out>, layermaps=..., prefilter=<optimized out>, postfilter=<optimized out>, attribute_accum=<optimized out>, filter=<optimized out>) at tile.cpp:2857
#3  0x00005645c7cdf1f4 in read_input (sources=..., fname=<optimized out>, maxzoom=<optimized out>, minzoom=<optimized out>, basezoom=<optimized out>, basezoom_marker_width=<optimized out>, outdb=<optimized out>, outdir=<optimized out>, exclude=<optimized out>, include=<optimized out>, exclude_all=<optimized out>, filter=<optimized out>, droprate=<optimized out>, buffer=<optimized out>, tmpdir=<optimized out>, gamma=<optimized out>, read_parallel=<optimized out>, forcetable=<optimized out>, attribution=<optimized out>, uses_gamma=<optimized out>, file_bbox=<optimized out>, prefilter=<optimized out>, postfilter=<optimized out>, description=<optimized out>, guess_maxzoom=<optimized out>, attribute_types=<optimized out>, pgm=<optimized out>, attribute_accum=<optimized out>, attribute_descriptions=..., commandline=...) at main.cpp:2270
#4  0x00005645c7c6aa1e in main (argc=<optimized out>, argv=0x7ffe69de8de8) at main.cpp:3183

Thread 8 (Thread 0x7fc3a858d700 (LWP 30674)):
#0  0x00007fc4798a9050 in ?? () from /lib/x86_64-linux-gnu/libz.so.1
#1  0x00007fc4798aaf98 in ?? () from /lib/x86_64-linux-gnu/libz.so.1
#2  0x00007fc4798ac1ed in deflate () from /lib/x86_64-linux-gnu/libz.so.1
#3  0x00005645c7ccae78 in compress (input="\032\357\333\027x\002\n\aparcels(\200 \032\nCountyFIPS\032\bATTOM ID\032\003APN\"\a\n\005\061\067\060\063\061\"\324\003\n\321\003\061\067\065\063\071\067\071\071\070,175398001,38953317,175397945,175397946,175394271,175403029,176078222,175908302,155105428,39004835,174897369,174897370,174897371,49"..., output="\037\213\b\000\000\000\000\000\002\003l\275}\350\377\313r\037\304\071\347\236s\356\375\334$7\017m-\027J\313\347/ů\272\063\263\217\224\326\304\346\201H\215\001\373O\251\bR\202\265\212\212\017\324\374W\255\210P\020\305H\244 b\251\004\n\202i\323H\203QӦ-\242\064\066\372\207R\005\021\"\221j[\251\264\330bug\037f\367\065\357{\370q\276\063\237\367>\314\316\356\316\316\316\316\316~\363\257\374ſ\343_\376\364\365\345?\367\217\377\363\277\373\307\376\351\177\341\357\374\375\277雯\337\366\317\376K\377̿\370\343?\370\303?\372\217|\363\253\337\367;~\307?\374\017\375\246\037\376\376o~\366}?\372#\357/_\237S\tB"...) at mvt.cpp:100
#4  0x00005645c7c83e27 in write_tile (geoms=geoms@entry=0x7fb02800fb70, geompos_in=geompos_in@entry=0x7fc3a858ceb0, metabase=<optimized out>, stringpool=<optimized out>, z=<optimized out>, tx=<optimized out>, ty=<optimized out>, detail=<optimized out>, min_detail=<optimized out>, outdb=<optimized out>, outdir=<optimized out>, buffer=<optimized out>, fname=<optimized out>, geomfile=<optimized out>, minzoom=<optimized out>, maxzoom=<optimized out>, todo=<optimized out>, along=<optimized out>, alongminus=<optimized out>, gamma=<optimized out>, child_shards=<optimized out>, meta_off=<optimized out>, pool_off=<optimized out>, initial_x=<optimized out>, initial_y=<optimized out>, running=<optimized out>, simplification=<optimized out>, layermaps=<optimized out>, layer_unmaps=<optimized out>, tiling_seg=<optimized out>, pass=<optimized out>, passes=<optimized out>, mingap=<optimized out>, minextent=<optimized out>, fraction=<optimized out>, prefilter=<optimized out>, postfilter=<optimized out>, filter=<optimized out>, arg=<optimized out>) at tile.cpp:2422
#5  0x00005645c7c86046 in run_thread (vargs=0x5645c8dc9cb0) at tile.cpp:2585
#6  0x00007fc4793856db in start_thread (arg=0x7fc3a858d700) at pthread_create.c:463
#7  0x00007fc47876f88f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 7 (Thread 0x7fc3a8d8e700 (LWP 30673)):
#0  0x00007fc4786e520a in tcache_get (tc_idx=28) at malloc.c:2943
#1  __GI___libc_malloc (bytes=464) at malloc.c:3050
#2  0x00007fc479088258 in operator new(unsigned long) () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#3  0x00005645c7c86ecb in __gnu_cxx::new_allocator<draw>::allocate (this=0x7fb034537148, __n=29) at /usr/include/c++/7/ext/new_allocator.h:111
#4  std::allocator_traits<std::allocator<draw> >::allocate (__a=..., __n=29) at /usr/include/c++/7/bits/alloc_traits.h:436
#5  std::_Vector_base<draw, std::allocator<draw> >::_M_allocate (this=0x7fb034537148, __n=29) at /usr/include/c++/7/bits/stl_vector.h:172
#6  std::_Vector_base<draw, std::allocator<draw> >::_M_create_storage (__n=<optimized out>, this=0x7fb034537148) at /usr/include/c++/7/bits/stl_vector.h:187
#7  std::_Vector_base<draw, std::allocator<draw> >::_Vector_base (__a=..., __n=<optimized out>, this=0x7fb034537148) at /usr/include/c++/7/bits/stl_vector.h:138
#8  std::vector<draw, std::allocator<draw> >::vector (this=0x7fb034537148, __x=std::vector of length 29, capacity 29 = {...}) at /usr/include/c++/7/bits/stl_vector.h:328
#9  0x00005645c7c8d7b8 in coalesce::coalesce (this=0x7fb0345370e0) at tile.cpp:84
#10 0x00005645c7c8205c in __gnu_cxx::new_allocator<coalesce>::construct<coalesce, coalesce const&> (this=0x7fb034007000, __p=<optimized out>) at /usr/include/c++/7/ext/new_allocator.h:136
#11 std::allocator_traits<std::allocator<coalesce> >::construct<coalesce, coalesce const&> (__a=..., __p=<optimized out>) at /usr/include/c++/7/bits/alloc_traits.h:475
#12 std::vector<coalesce, std::allocator<coalesce> >::push_back (__x=..., this=0x7fb034007000) at /usr/include/c++/7/bits/stl_vector.h:943
#13 write_tile (geoms=geoms@entry=0x7fb0341fb4a0, geompos_in=geompos_in@entry=0x7fc3a8d8deb0, metabase=<optimized out>, stringpool=<optimized out>, z=<optimized out>, tx=<optimized out>, ty=<optimized out>, detail=<optimized out>, min_detail=<optimized out>, outdb=<optimized out>, outdir=<optimized out>, buffer=<optimized out>, fname=<optimized out>, geomfile=<optimized out>, minzoom=<optimized out>, maxzoom=<optimized out>, todo=<optimized out>, along=<optimized out>, alongminus=<optimized out>, gamma=<optimized out>, child_shards=<optimized out>, meta_off=<optimized out>, pool_off=<optimized out>, initial_x=<optimized out>, initial_y=<optimized out>, running=<optimized out>, simplification=<optimized out>, layermaps=<optimized out>, layer_unmaps=<optimized out>, tiling_seg=<optimized out>, pass=<optimized out>, passes=<optimized out>, mingap=<optimized out>, minextent=<optimized out>, fraction=<optimized out>, prefilter=<optimized out>, postfilter=<optimized out>, filter=<optimized out>, arg=<optimized out>) at tile.cpp:2169
#14 0x00005645c7c86046 in run_thread (vargs=0x5645c8dc9b50) at tile.cpp:2585
#15 0x00007fc4793856db in start_thread (arg=0x7fc3a8d8e700) at pthread_create.c:463
#16 0x00007fc47876f88f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 6 (Thread 0x7fc3a5d88700 (LWP 30676)):
#0  0x00005645c7cb745e in std::__adjacent_find<__gnu_cxx::__normal_iterator<long long*, std::vector<long long, std::allocator<long long> > >, __gnu_cxx::__ops::_Iter_equal_to_iter> (__last=..., __first=63, __binary_pred=...) at /usr/include/c++/7/bits/stl_algo.h:953
#1  std::__unique<__gnu_cxx::__normal_iterator<long long*, std::vector<long long, std::allocator<long long> > >, __gnu_cxx::__ops::_Iter_equal_to_iter> (__binary_pred=..., __last=681, __first=31) at /usr/include/c++/7/bits/stl_algo.h:966
#2  std::unique<__gnu_cxx::__normal_iterator<long long*, std::vector<long long, std::allocator<long long> > > > (__last=..., __first=...) at /usr/include/c++/7/bits/stl_algo.h:1005
#3  mapbox::geometry::wagyu::pop_from_scanbeam<long long> (Y=@0x7fc3a5d86cc0: 668, scanbeam=std::vector of length 74, capacity 256 = {...}) at ./mapbox/geometry/wagyu/scanbeam.hpp:21
#4  0x00005645c7cbdeb2 in mapbox::geometry::wagyu::build_hot_pixels<long long> (minima_list=std::deque with 156 elements = {...}, manager=...) at ./mapbox/geometry/wagyu/snap_rounding.hpp:181
#5  0x00005645c7ca3c5b in mapbox::geometry::wagyu::wagyu<long long>::execute<long long> (clip_fill_type=mapbox::geometry::wagyu::fill_type_positive, subject_fill_type=mapbox::geometry::wagyu::fill_type_positive, solution=..., cliptype=mapbox::geometry::wagyu::clip_type_union, this=0x7fc3a5d86e20) at ./mapbox/geometry/wagyu/wagyu.hpp:128
#6  clean_or_clip_poly (geom=std::vector of length 791, capacity 1024 = {...}, z=z@entry=0, buffer=buffer@entry=0, clip=clip@entry=false) at geometry.cpp:272
#7  0x00005645c7c7658c in partial_feature_worker (v=0x7fb01c0a84f0) at tile.cpp:501
#8  0x00005645c7c844ee in write_tile (geoms=geoms@entry=0x7fb01c03a2e0, geompos_in=geompos_in@entry=0x7fc3a5d87eb0, metabase=<optimized out>, stringpool=<optimized out>, z=<optimized out>, tx=<optimized out>, ty=<optimized out>, detail=<optimized out>, min_detail=<optimized out>, outdb=<optimized out>, outdir=<optimized out>, buffer=<optimized out>, fname=<optimized out>, geomfile=<optimized out>, minzoom=<optimized out>, maxzoom=<optimized out>, todo=<optimized out>, along=<optimized out>, alongminus=<optimized out>, gamma=<optimized out>, child_shards=<optimized out>, meta_off=<optimized out>, pool_off=<optimized out>, initial_x=<optimized out>, initial_y=<optimized out>, running=<optimized out>, simplification=<optimized out>, layermaps=<optimized out>, layer_unmaps=<optimized out>, tiling_seg=<optimized out>, pass=<optimized out>, passes=<optimized out>, mingap=<optimized out>, minextent=<optimized out>, fraction=<optimized out>, prefilter=<optimized out>, postfilter=<optimized out>, filter=<optimized out>, arg=<optimized out>) at tile.cpp:2115
#9  0x00005645c7c86046 in run_thread (vargs=0x5645c8dc9f70) at tile.cpp:2585
#10 0x00007fc4793856db in start_thread (arg=0x7fc3a5d88700) at pthread_create.c:463
#11 0x00007fc47876f88f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 5 (Thread 0x7fc3a958f700 (LWP 30672)):
#0  0x00005645c7caf95a in mapbox::geometry::wagyu::round_towards_min<long long> (val=1132.8500000000211) at ./mapbox/geometry/wagyu/ring_util.hpp:91
#1  mapbox::geometry::wagyu::get_edge_min_x<long long> (current_y=2974, edge=...) at ./mapbox/geometry/wagyu/ring_util.hpp:126
#2  mapbox::geometry::wagyu::hot_pixel_set_left_to_right<long long> (add_end_point=false, end=<synthetic pointer>{x = 954, y = 3299}, itr=Python Exception <class 'gdb.error'> value has been optimized out:
<synthetic pointer>, rings=..., bnd=..., end_x=1154, start_x=731, y=2974) at ./mapbox/geometry/wagyu/ring_util.hpp:171
#3  mapbox::geometry::wagyu::insert_hot_pixels_in_path<long long> (bnd=..., end_pt=..., rings=..., add_end_point=add_end_point@entry=false) at ./mapbox/geometry/wagyu/ring_util.hpp:296
#4  0x00005645c7cafc74 in mapbox::geometry::wagyu::add_point_to_ring<long long> (bnd=..., pt=..., rings=...) at ./mapbox/geometry/wagyu/ring_util.hpp:327
#5  0x00005645c7cbe985 in mapbox::geometry::wagyu::add_point<long long> (bnd=..., active_bounds=std::vector of length 2, capacity 4 = {...}, pt=..., rings=...) at ./mapbox/geometry/wagyu/ring_util.hpp:352
#6  0x00005645c7cbebb8 in mapbox::geometry::wagyu::add_local_maximum_point<long long> (b1=..., b2=..., pt=..., rings=..., active_bounds=std::vector of length 2, capacity 4 = {...}) at ./mapbox/geometry/wagyu/ring_util.hpp:623
#7  0x00005645c7cc0c7c in mapbox::geometry::wagyu::do_maxima<long long> (active_bounds=std::vector of length 2, capacity 4 = {...}, manager=..., clip_fill_type=mapbox::geometry::wagyu::fill_type_positive, subject_fill_type=mapbox::geometry::wagyu::fill_type_positive, cliptype=mapbox::geometry::wagyu::clip_type_union, bndMaxPair=Python Exception <class 'gdb.error'> value has been optimized out:
<synthetic pointer>, bnd=0x7fb02c01d920) at ./mapbox/geometry/wagyu/process_maxima.hpp:44
#8  mapbox::geometry::wagyu::process_edges_at_top_of_scanbeam<long long> (top_y=<optimized out>, active_bounds=std::vector of length 2, capacity 4 = {...}, scanbeam=std::vector of length 0, capacity 8, minima_sorted=std::vector of length 4, capacity 4 = {...}, current_lm=0xc040b33333333333, manager=..., cliptype=mapbox::geometry::wagyu::clip_type_union, subject_fill_type=mapbox::geometry::wagyu::fill_type_positive, clip_fill_type=mapbox::geometry::wagyu::fill_type_positive) at ./mapbox/geometry/wagyu/process_maxima.hpp:84
#9  0x00005645c7cc1016 in mapbox::geometry::wagyu::execute_vatti<long long> (minima_list=std::deque with 4 elements = {...}, manager=..., cliptype=cliptype@entry=mapbox::geometry::wagyu::clip_type_union, subject_fill_type=subject_fill_type@entry=mapbox::geometry::wagyu::fill_type_positive, clip_fill_type=clip_fill_type@entry=mapbox::geometry::wagyu::fill_type_positive) at ./mapbox/geometry/wagyu/vatti.hpp:53
#10 0x00005645c7ca3c7d in mapbox::geometry::wagyu::wagyu<long long>::execute<long long> (clip_fill_type=mapbox::geometry::wagyu::fill_type_positive, subject_fill_type=mapbox::geometry::wagyu::fill_type_positive, solution=..., cliptype=mapbox::geometry::wagyu::clip_type_union, this=0x7fc3a958de20) at ./mapbox/geometry/wagyu/wagyu.hpp:130
#11 clean_or_clip_poly (geom=std::vector of length 24, capacity 32 = {...}, z=z@entry=0, buffer=buffer@entry=0, clip=clip@entry=false) at geometry.cpp:272
#12 0x00005645c7c7658c in partial_feature_worker (v=0x7fb02c056b00) at tile.cpp:501
#13 0x00005645c7c844ee in write_tile (geoms=geoms@entry=0x7fb02c09b9c0, geompos_in=geompos_in@entry=0x7fc3a958eeb0, metabase=<optimized out>, stringpool=<optimized out>, z=<optimized out>, tx=<optimized out>, ty=<optimized out>, detail=<optimized out>, min_detail=<optimized out>, outdb=<optimized out>, outdir=<optimized out>, buffer=<optimized out>, fname=<optimized out>, geomfile=<optimized out>, minzoom=<optimized out>, maxzoom=<optimized out>, todo=<optimized out>, along=<optimized out>, alongminus=<optimized out>, gamma=<optimized out>, child_shards=<optimized out>, meta_off=<optimized out>, pool_off=<optimized out>, initial_x=<optimized out>, initial_y=<optimized out>, running=<optimized out>, simplification=<optimized out>, layermaps=<optimized out>, layer_unmaps=<optimized out>, tiling_seg=<optimized out>, pass=<optimized out>, passes=<optimized out>, mingap=<optimized out>, minextent=<optimized out>, fraction=<optimized out>, prefilter=<optimized out>, postfilter=<optimized out>, filter=<optimized out>, arg=<optimized out>) at tile.cpp:2115
#14 0x00005645c7c86046 in run_thread (vargs=0x5645c8dc99f0) at tile.cpp:2585
#15 0x00007fc4793856db in start_thread (arg=0x7fc3a958f700) at pthread_create.c:463
#16 0x00007fc47876f88f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 4 (Thread 0x7fc3a6589700 (LWP 30675)):
#0  std::__move_merge_adaptive_backward<__gnu_cxx::__normal_iterator<mapbox::geometry::wagyu::local_minimum<long long>**, std::vector<mapbox::geometry::wagyu::local_minimum<long long>*, std::allocator<mapbox::geometry::wagyu::local_minimum<long long>*> > >, mapbox::geometry::wagyu::local_minimum<long long>**, __gnu_cxx::__normal_iterator<mapbox::geometry::wagyu::local_minimum<long long>**, std::vector<mapbox::geometry::wagyu::local_minimum<long long>*, std::allocator<mapbox::geometry::wagyu::local_minimum<long long>*> > >, __gnu_cxx::__ops::_Iter_comp_iter<mapbox::geometry::wagyu::local_minimum_sorter<long long> > > (__comp=..., __result=0x7fb03c9ece80, __last2=0x7fb03c3c61a0, __first2=0x7fb03c3c6190, __last1=0x7fb03c1bbf60, __first1=...) at /usr/include/c++/7/bits/stl_algo.h:2362
#1  std::__merge_adaptive<__gnu_cxx::__normal_iterator<mapbox::geometry::wagyu::local_minimum<long long>**, std::vector<mapbox::geometry::wagyu::local_minimum<long long>*, std::allocator<mapbox::geometry::wagyu::local_minimum<long long>*> > >, long, mapbox::geometry::wagyu::local_minimum<long long>**, __gnu_cxx::__ops::_Iter_comp_iter<mapbox::geometry::wagyu::local_minimum_sorter<long long> > > (__first=0x7fb03c3c5680, __middle=..., __last=0x7fb03c223610, __len1=<optimized out>, __len2=12, __buffer=0x7fb03c3c6190, __buffer_size=25, __comp=...) at /usr/include/c++/7/bits/stl_algo.h:2431
#2  0x00005645c7cb2add in std::__stable_sort_adaptive<__gnu_cxx::__normal_iterator<mapbox::geometry::wagyu::local_minimum<long long>**, std::vector<mapbox::geometry::wagyu::local_minimum<long long>*, std::allocator<mapbox::geometry::wagyu::local_minimum<long long>*> > >, mapbox::geometry::wagyu::local_minimum<long long>**, long, __gnu_cxx::__ops::_Iter_comp_iter<mapbox::geometry::wagyu::local_minimum_sorter<long long> > > (__first=0x7fb03c9ece80, __last=<error reading variable: Cannot access memory at address 0x770>, __last@entry=0x7fb03c223610, __buffer=__buffer@entry=0x7fb03c3c6190, __buffer_size=__buffer_size@entry=25, __comp=...) at /usr/include/c++/7/bits/stl_algo.h:2751
#3  0x00005645c7cc18a8 in std::__stable_sort<__gnu_cxx::__normal_iterator<mapbox::geometry::wagyu::local_minimum<long long>**, std::vector<mapbox::geometry::wagyu::local_minimum<long long>*, std::allocator<mapbox::geometry::wagyu::local_minimum<long long>*> > >, __gnu_cxx::__ops::_Iter_comp_iter<mapbox::geometry::wagyu::local_minimum_sorter<long long> > > (__comp=..., __last=..., __first=...) at /usr/include/c++/7/bits/stl_algo.h:5008
#4  std::stable_sort<__gnu_cxx::__normal_iterator<mapbox::geometry::wagyu::local_minimum<long long>**, std::vector<mapbox::geometry::wagyu::local_minimum<long long>*, std::allocator<mapbox::geometry::wagyu::local_minimum<long long>*> > >, mapbox::geometry::wagyu::local_minimum_sorter<long long> > (__last=..., __first=..., __comp=...) at /usr/include/c++/7/bits/stl_algo.h:5077
#5  mapbox::geometry::wagyu::execute_vatti<long long> (minima_list=std::deque with 25 elements = {...}, manager=..., cliptype=cliptype@entry=mapbox::geometry::wagyu::clip_type_union, subject_fill_type=subject_fill_type@entry=mapbox::geometry::wagyu::fill_type_positive, clip_fill_type=clip_fill_type@entry=mapbox::geometry::wagyu::fill_type_positive) at ./mapbox/geometry/wagyu/vatti.hpp:36
#6  0x00005645c7ca3c7d in mapbox::geometry::wagyu::wagyu<long long>::execute<long long> (clip_fill_type=mapbox::geometry::wagyu::fill_type_positive, subject_fill_type=mapbox::geometry::wagyu::fill_type_positive, solution=..., cliptype=mapbox::geometry::wagyu::clip_type_union, this=0x7fc3a6587e20) at ./mapbox/geometry/wagyu/wagyu.hpp:130
#7  clean_or_clip_poly (geom=std::vector of length 120, capacity 128 = {...}, z=z@entry=0, buffer=buffer@entry=0, clip=clip@entry=false) at geometry.cpp:272
#8  0x00005645c7c7658c in partial_feature_worker (v=0x7fb03d494b70) at tile.cpp:501
#9  0x00005645c7c844ee in write_tile (geoms=geoms@entry=0x7fb03c27c710, geompos_in=geompos_in@entry=0x7fc3a6588eb0, metabase=<optimized out>, stringpool=<optimized out>, z=<optimized out>, tx=<optimized out>, ty=<optimized out>, detail=<optimized out>, min_detail=<optimized out>, outdb=<optimized out>, outdir=<optimized out>, buffer=<optimized out>, fname=<optimized out>, geomfile=<optimized out>, minzoom=<optimized out>, maxzoom=<optimized out>, todo=<optimized out>, along=<optimized out>, alongminus=<optimized out>, gamma=<optimized out>, child_shards=<optimized out>, meta_off=<optimized out>, pool_off=<optimized out>, initial_x=<optimized out>, initial_y=<optimized out>, running=<optimized out>, simplification=<optimized out>, layermaps=<optimized out>, layer_unmaps=<optimized out>, tiling_seg=<optimized out>, pass=<optimized out>, passes=<optimized out>, mingap=<optimized out>, minextent=<optimized out>, fraction=<optimized out>, prefilter=<optimized out>, postfilter=<optimized out>, filter=<optimized out>, arg=<optimized out>) at tile.cpp:2115
#10 0x00005645c7c86046 in run_thread (vargs=0x5645c8dc9e10) at tile.cpp:2585
#11 0x00007fc4793856db in start_thread (arg=0x7fc3a6589700) at pthread_create.c:463
#12 0x00007fc47876f88f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 3 (Thread 0x7fc3a758b700 (LWP 30670)):
#0  0x00007fc4798a907c in ?? () from /lib/x86_64-linux-gnu/libz.so.1
#1  0x00007fc4798aaf98 in ?? () from /lib/x86_64-linux-gnu/libz.so.1
#2  0x00007fc4798ac1ed in deflate () from /lib/x86_64-linux-gnu/libz.so.1
#3  0x00005645c7ccae78 in compress (input="\032\224\315\021x\002\n\aparcels(\200 \032\nCountyFIPS\032\bATTOM ID\032\003APN\"\a\n\005\064\070\063\067\065\"[\nY165089343,244128855,164887659,166242411,244128854,244569663,245221457,167882610,251930128\"\177\n}R065162012000,R065162011900,R065162045680,R065162"..., output="\037\213\b\000\000\000\000\000\002\003d\275K\250\377\301\222\330\005wfr\357=\367\225;w&\213\001\061\234\225\342Ytw\365s\031\024!\213h\320lĕ\210;Q\361\001\272\020EA\204\370\b\030P\f1&DF\006$\032\334J$\v%\276 00\340#F\207Ġ\213HF\023#\001\355\252\356\256\356\317?gq~]\337~W\277\252\252\253\252\177\345\017\376\267?\375g\277\363\361\335\177\342\037\372'\377\341\177\344\037\375\247\376\266\177\361w\376\312\307\337\371\217\377\063\377\330?\375\317\375ݿ\373\367\376\375\277\362\275\337\365\373~\337\337\373{~\347\357\376\273~\345\347~\327\357\375{>\277\373\361\v\271K+\237\377\340\307?\020k\t"...) at mvt.cpp:100
#4  0x00005645c7c83e27 in write_tile (geoms=geoms@entry=0x7fb0384ba820, geompos_in=geompos_in@entry=0x7fc3a758aeb0, metabase=<optimized out>, stringpool=<optimized out>, z=<optimized out>, tx=<optimized out>, ty=<optimized out>, detail=<optimized out>, min_detail=<optimized out>, outdb=<optimized out>, outdir=<optimized out>, buffer=<optimized out>, fname=<optimized out>, geomfile=<optimized out>, minzoom=<optimized out>, maxzoom=<optimized out>, todo=<optimized out>, along=<optimized out>, alongminus=<optimized out>, gamma=<optimized out>, child_shards=<optimized out>, meta_off=<optimized out>, pool_off=<optimized out>, initial_x=<optimized out>, initial_y=<optimized out>, running=<optimized out>, simplification=<optimized out>, layermaps=<optimized out>, layer_unmaps=<optimized out>, tiling_seg=<optimized out>, pass=<optimized out>, passes=<optimized out>, mingap=<optimized out>, minextent=<optimized out>, fraction=<optimized out>, prefilter=<optimized out>, postfilter=<optimized out>, filter=<optimized out>, arg=<optimized out>) at tile.cpp:2422
#5  0x00005645c7c86046 in run_thread (vargs=0x5645c8dc9730) at tile.cpp:2585
#6  0x00007fc4793856db in start_thread (arg=0x7fc3a758b700) at pthread_create.c:463
#7  0x00007fc47876f88f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 2 (Thread 0x7fc3a7d8c700 (LWP 30671)):
#0  0x00007fc4786d5df2 in _IO_getc (fp=0x7fb0300737e0) at getc.c:39
#1  0x00005645c7cd3592 in deserialize_ulong_long_io (geompos=0x7fc3a7d8beb0, zigzag=<synthetic pointer>, f=0x7fb0300737e0) at serial.cpp:132
#2  deserialize_long_long_io (f=f@entry=0x7fb0300737e0, n=n@entry=0x7fc3a7d8aba0, geompos=geompos@entry=0x7fc3a7d8beb0) at serial.cpp:122
#3  0x00005645c7c9e4e8 in decode_geometry (meta=meta@entry=0x7fb0300737e0, geompos=geompos@entry=0x7fc3a7d8beb0, z=z@entry=13, tx=tx@entry=2401, ty=ty@entry=3076, bbox=0x7fc3a7d8afd8, initial_x=733864672, initial_y=1714676128) at geometry.cpp:51
#4  0x00005645c7cd4370 in deserialize_feature (geoms=0x7fb0300737e0, geompos_in=0x7fc3a7d8beb0, metabase=0x7fc477f7f000 <error: Cannot access memory at address 0x7fc477f7f000>, meta_off=0x7ffe69de6ff0, z=13, tx=2401, ty=3076, initial_x=0x7ffe69de7160, initial_y=0x7ffe69de7110) at serial.cpp:279
#5  0x00005645c7c76c0f in next_feature (geoms=geoms@entry=0x7fb0300737e0, geompos_in=geompos_in@entry=0x7fc3a7d8beb0, metabase=metabase@entry=0x7fc477f7f000 <error: Cannot access memory at address 0x7fc477f7f000>, meta_off=meta_off@entry=0x7ffe69de6ff0, z=z@entry=13, tx=<optimized out>, ty=<optimized out>, initial_x=<optimized out>, initial_y=<optimized out>, original_features=<optimized out>, unclipped_features=<optimized out>, nextzoom=<optimized out>, maxzoom=<optimized out>, minzoom=<optimized out>, max_zoom_increment=<optimized out>, pass=<optimized out>, passes=<optimized out>, along=<optimized out>, alongminus=<optimized out>, buffer=<optimized out>, within=<optimized out>, first_time=<optimized out>, geomfile=<optimized out>, geompos=<optimized out>, oprogress=<optimized out>, todo=todo@entry=12998412398, fname=<optimized out>, child_shards=<optimized out>, filter=<optimized out>, stringpool=<optimized out>, pool_off=<optimized out>, layer_unmaps=<optimized out>) at tile.cpp:1317
#6  0x00005645c7c80c8d in write_tile (geoms=geoms@entry=0x7fb0300737e0, geompos_in=geompos_in@entry=0x7fc3a7d8beb0, metabase=<optimized out>, stringpool=<optimized out>, z=<optimized out>, tx=<optimized out>, ty=<optimized out>, detail=<optimized out>, min_detail=<optimized out>, outdb=<optimized out>, outdir=<optimized out>, buffer=<optimized out>, fname=<optimized out>, geomfile=<optimized out>, minzoom=<optimized out>, maxzoom=<optimized out>, todo=<optimized out>, along=<optimized out>, alongminus=<optimized out>, gamma=<optimized out>, child_shards=<optimized out>, meta_off=<optimized out>, pool_off=<optimized out>, initial_x=<optimized out>, initial_y=<optimized out>, running=<optimized out>, simplification=<optimized out>, layermaps=<optimized out>, layer_unmaps=<optimized out>, tiling_seg=<optimized out>, pass=<optimized out>, passes=<optimized out>, mingap=<optimized out>, minextent=<optimized out>, fraction=<optimized out>, prefilter=<optimized out>, postfilter=<optimized out>, filter=<optimized out>, arg=<optimized out>) at tile.cpp:1861
#7  0x00005645c7c86046 in run_thread (vargs=0x5645c8dc9890) at tile.cpp:2585
#8  0x00007fc4793856db in start_thread (arg=0x7fc3a7d8c700) at pthread_create.c:463
#9  0x00007fc47876f88f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 1 (Thread 0x7fc3a6d8a700 (LWP 30669)):
#0  deserialize_ulong_long (zigzag=<synthetic pointer>, f=<synthetic pointer>) at serial.cpp:97
#1  deserialize_long_long (n=<synthetic pointer>, f=<synthetic pointer>) at serial.cpp:88
#2  deserialize_feature (geoms=0x7fb024138d70, geompos_in=0x7fc3a6d89eb0, metabase=0x7fc477f7f000 <error: Cannot access memory at address 0x7fc477f7f000>, meta_off=0x7ffe69de6ff0, z=4294967249, tx=2197946775, ty=33667841, initial_x=0x7ffe69de7160, initial_y=0x7ffe69de7110) at serial.cpp:295
#3  0x00005645c7c76c0f in next_feature (geoms=geoms@entry=0x7fb024138d70, geompos_in=geompos_in@entry=0x7fc3a6d89eb0, metabase=metabase@entry=0x7fc477f7f000 <error: Cannot access memory at address 0x7fc477f7f000>, meta_off=meta_off@entry=0x7ffe69de6ff0, z=z@entry=-47, tx=<optimized out>, ty=<optimized out>, initial_x=<optimized out>, initial_y=<optimized out>, original_features=<optimized out>, unclipped_features=<optimized out>, nextzoom=<optimized out>, maxzoom=<optimized out>, minzoom=<optimized out>, max_zoom_increment=<optimized out>, pass=<optimized out>, passes=<optimized out>, along=<optimized out>, alongminus=<optimized out>, buffer=<optimized out>, within=<optimized out>, first_time=<optimized out>, geomfile=<optimized out>, geompos=<optimized out>, oprogress=<optimized out>, todo=todo@entry=12998412398, fname=<optimized out>, child_shards=<optimized out>, filter=<optimized out>, stringpool=<optimized out>, pool_off=<optimized out>, layer_unmaps=<optimized out>) at tile.cpp:1317
#4  0x00005645c7c80c8d in write_tile (geoms=geoms@entry=0x7fb024138d70, geompos_in=geompos_in@entry=0x7fc3a6d89eb0, metabase=<optimized out>, stringpool=<optimized out>, z=<optimized out>, tx=<optimized out>, ty=<optimized out>, detail=<optimized out>, min_detail=<optimized out>, outdb=<optimized out>, outdir=<optimized out>, buffer=<optimized out>, fname=<optimized out>, geomfile=<optimized out>, minzoom=<optimized out>, maxzoom=<optimized out>, todo=<optimized out>, along=<optimized out>, alongminus=<optimized out>, gamma=<optimized out>, child_shards=<optimized out>, meta_off=<optimized out>, pool_off=<optimized out>, initial_x=<optimized out>, initial_y=<optimized out>, running=<optimized out>, simplification=<optimized out>, layermaps=<optimized out>, layer_unmaps=<optimized out>, tiling_seg=<optimized out>, pass=<optimized out>, passes=<optimized out>, mingap=<optimized out>, minextent=<optimized out>, fraction=<optimized out>, prefilter=<optimized out>, postfilter=<optimized out>, filter=<optimized out>, arg=<optimized out>) at tile.cpp:1861
#5  0x00005645c7c86046 in run_thread (vargs=0x5645c8dc95d0) at tile.cpp:2585
#6  0x00007fc4793856db in start_thread (arg=0x7fc3a6d8a700) at pthread_create.c:463
#7  0x00007fc47876f88f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
s

With this command line the crash did not occur:

~/logbt/logbt -- tippecanoe --output=mbtiles/parcels.mbtiles --force --read-parallel --minimum-zoom=13 --maximum-zoom=15 --generate-ids --simplification=10 --simplify-only-low-zooms --include=APN --include=CountyFIPS --coalesce-smallest-as-needed --base-zoom=14 --accumulate-attribute=APN:comma --layer=parcels data/parcels.json

springmeyer commented 4 years ago

Thanks @stevage - this is the key line that appears to be causing the segfault:

deserialize_ulong_long (zigzag=, f=) at serial.cpp:97

@ericfischer any idea how to fix this?

stevage commented 4 years ago

I'd also be interested to hear any possible workarounds.

e-n-f commented 4 years ago

Thanks. This must be indexing off the end of the buffer that encodes a variable-length number, either because it was encoded wrong in the first place, or because the metadata pointer is pointing to the wrong place.

I wish I had a workaround to offer immediately, but it looks like it's not as easy as I thought to substitute a different representation for serialized numbers. I'll keep investigating.

e-n-f commented 4 years ago

The workaround seems to be to insert inline_meta = true before this check:

https://github.com/mapbox/tippecanoe/blob/ddb79937d932f753edd5fba994b23281ff45f19c/serial.cpp#L654

but I'm still not sure what is causing the externalized metadata to be misencoded.

stevage commented 4 years ago

Is there anything more I can do to help?

stevage commented 4 years ago

OIC. I'll try patching that and see if it prevents the crash.

Sorry for the dumb question, but will there be side-effects from that change? I'm not really sure what "externalized metadata" is.

e-n-f commented 4 years ago

Sorry to be unclear. The "separate metadata" file is basically a holdover from an older version of Tippecanoe in which most of the characteristics of a feature were stored in a separate file so they wouldn't have to be duplicated across temporary files when a feature was clipped into many sub-features in different tiles. I gradually moved more and more things into the "geometry" file, so all it has in it now is a table of indices of key/value attribute pairs into the attribute pool, and is only populated when a feature is expected to span a large number of tiles.

I should probably just get rid of it, since it doesn't seem to work reliably and is another thing that has to be handled as a special case when it exists.

e-n-f commented 4 years ago

Could you please try https://github.com/mapbox/tippecanoe/pull/845 and see if it fixes the problem for you?

stevage commented 4 years ago

It crashed as before:

Going to try keeping the biggest 20.25% of the features to make it fit
Segmentation fault073035818/2695646785
Tiles for parcels:parcels failed in 4.14 h.

I forgot to use logbt, so will do that now.

e-n-f commented 4 years ago

Hmm, so much for that idea. Any chance you could share your data file so I can try to reproduce the exact circumstances?

stevage commented 4 years ago

Trying to find a way to do that. It's 81GB uncompressed, and also subject to licensing rules which I'm trying to get clarity on.

In the meantime, here's logbt:

mfile=0x7fff97437b60, fname=0x7fff9743bfda "mbtiles/parcels.mbtiles", t=<optimized out>, layer=<optimized out>, feature_minzoom=<optimized out>, child_shards=<optimized out>, max_zoom_increment=<optimized out>, seq=0, tippecanoe_minzoom=<optimized out>, tippecanoe_maxzoom=<optimized out>, segment=<optimized out>, initial_x=0x7fff97438240, initial_y=0x7fff974381f0, metakeys=std::vector of length 3, capacity 4 = {...}, metavals=std::vector of length 3, capacity 4 = {...}, has_id=true, id=21697592808, index=0, extent=9824) at tile.cpp:348
#6  0x00005594d9a1fa35 in next_feature (geoms=geoms@entry=0x7f41245b7100, geompos_in=geompos_in@entry=0x7f54a5ae2eb0, z=z@entry=13, tx=tx@entry=1416, ty=ty@entry=3285, initial_x=<optimized out>, initial_y=<optimized out>, original_features=<optimized out>, unclipped_features=<optimized out>, nextzoom=<optimized out>, maxzoom=<optimized out>, minzoom=<optimized out>, max_zoom_increment=<optimized out>, pass=<optimized out>, passes=<optimized out>, along=<optimized out>, alongminus=<optimized out>, buffer=<optimized out>, within=<optimized out>, first_time=<optimized out>, geomfile=<optimized out>, geompos=<optimized out>, oprogress=<optimized out>, todo=todo@entry=12849808566, fname=<optimized out>, child_shards=<optimized out>, filter=<optimized out>, stringpool=<optimized out>, pool_off=<optimized out>, layer_unmaps=<optimized out>) at tile.cpp:1343
#7  0x00005594d9a28930 in write_tile (geoms=geoms@entry=0x7f41245b7100, geompos_in=geompos_in@entry=0x7f54a5ae2eb0, stringpool=<optimized out>, z=<optimized out>, tx=<optimized out>, ty=<optimized out>, detail=<optimized out>, min_detail=<optimized out>, outdb=<optimized out>, outdir=<optimized out>, buffer=<optimized out>, fname=<optimized out>, geomfile=<optimized out>, minzoom=<optimized out>, maxzoom=<optimized out>, todo=<optimized out>, along=<optimized out>, alongminus=<optimized out>, gamma=<optimized out>, child_shards=<optimized out>, pool_off=<optimized out>, initial_x=<optimized out>, initial_y=<optimized out>, running=<optimized out>, simplification=<optimized out>, layermaps=<optimized out>, layer_unmaps=<optimized out>, tiling_seg=<optimized out>, pass=<optimized out>, passes=<optimized out>, mingap=<optimized out>, minextent=<optimized out>, fraction=<optimized out>, prefilter=<optimized out>, postfilter=<optimized out>, filter=<optimized out>, arg=<optimized out>) at tile.cpp:1852
#8  0x00005594d9a2dd6a in run_thread (vargs=0x5594dad981d0) at tile.cpp:2576
#9  0x00007f5575c0f6db in start_thread (arg=0x7f54a5ae3700) at pthread_create.c:463
#10 0x00007f5574ff988f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 2 (Thread 0x7f54a32de700 (LWP 7280)):
#0  serialize_ulong_long (out=out@entry=0x5594dad96d50, zigzag=<optimized out>, fpos=fpos@entry=0x7f54a32dd330, fname=fname@entry=0x7fff9743bfda "mbtiles/parcels.mbtiles") at serial.cpp:68
#1  0x00005594d9a7bb46 in serialize_long_long (fname=0x7fff9743bfda "mbtiles/parcels.mbtiles", fpos=0x7f54a32dd330, n=<optimized out>, out=0x5594dad96d50) at serial.cpp:45
#2  write_geometry (wy=51470034, wx=38823941, fname=0x7fff9743bfda "mbtiles/parcels.mbtiles", out=0x5594dad96d50, fpos=0x7f54a32dd330, dv=std::vector of length 16, capacity 16 = {...}) at serial.cpp:180
#3  serialize_feature (geomfile=0x5594dad96d50, sf=sf@entry=0x7f54a32dccc0, geompos=geompos@entry=0x7f54a32dd330, fname=fname@entry=0x7fff9743bfda "mbtiles/parcels.mbtiles", wx=<optimized out>, wy=<optimized out>, include_minzoom=true) at serial.cpp:219
#4  0x00005594d9a1de65 in rewrite (geom=std::vector of length 16, capacity 16 = {...}, z=z@entry=13, nextzoom=14, maxzoom=<optimized out>, bbox=bbox@entry=0x7f54a32dd1a0, tx=tx@entry=2369, ty=<optimized out>, buffer=<optimized out>, within=0x7f54a32dd350, geompos=0x7f54a32dd320, geomfile=0x7fff97437ba0, fname=0x7fff9743bfda "mbtiles/parcels.mbtiles", t=<optimized out>, layer=<optimized out>, feature_minzoom=<optimized out>, child_shards=<optimized out>, max_zoom_increment=<optimized out>, seq=0, tippecanoe_minzoom=<optimized out>, tippecanoe_maxzoom=<optimized out>, segment=<optimized out>, initial_x=0x7fff97438240, initial_y=0x7fff974381f0, metakeys=std::vector of length 3, capacity 4 = {...}, metavals=std::vector of length 3, capacity 4 = {...}, has_id=true, id=10857074377, index=0, extent=42052) at tile.cpp:348
#5  0x00005594d9a1fa35 in next_feature (geoms=geoms@entry=0x7f41300084a0, geompos_in=geompos_in@entry=0x7f54a32ddeb0, z=z@entry=13, tx=tx@entry=2369, ty=ty@entry=3141, initial_x=<optimized out>, initial_y=<optimized out>, original_features=<optimized out>, unclipped_features=<optimized out>, nextzoom=<optimized out>, maxzoom=<optimized out>, minzoom=<optimized out>, max_zoom_increment=<optimized out>, pass=<optimized out>, passes=<optimized out>, along=<optimized out>, alongminus=<optimized out>, buffer=<optimized out>, within=<optimized out>, first_time=<optimized out>, geomfile=<optimized out>, geompos=<optimized out>, oprogress=<optimized out>, todo=todo@entry=12849808566, fname=<optimized out>, child_shards=<optimized out>, filter=<optimized out>, stringpool=<optimized out>, pool_off=<optimized out>, layer_unmaps=<optimized out>) at tile.cpp:1343
#6  0x00005594d9a28930 in write_tile (geoms=geoms@entry=0x7f41300084a0, geompos_in=geompos_in@entry=0x7f54a32ddeb0, stringpool=<optimized out>, z=<optimized out>, tx=<optimized out>, ty=<optimized out>, detail=<optimized out>, min_detail=<optimized out>, outdb=<optimized out>, outdir=<optimized out>, buffer=<optimized out>, fname=<optimized out>, geomfile=<optimized out>, minzoom=<optimized out>, maxzoom=<optimized out>, todo=<optimized out>, along=<optimized out>, alongminus=<optimized out>, gamma=<optimized out>, child_shards=<optimized out>, pool_off=<optimized out>, initial_x=<optimized out>, initial_y=<optimized out>, running=<optimized out>, simplification=<optimized out>, layermaps=<optimized out>, layer_unmaps=<optimized out>, tiling_seg=<optimized out>, pass=<optimized out>, passes=<optimized out>, mingap=<optimized out>, minextent=<optimized out>, fraction=<optimized out>, prefilter=<optimized out>, postfilter=<optimized out>, filter=<optimized out>, arg=<optimized out>) at tile.cpp:1852
#7  0x00005594d9a2dd6a in run_thread (vargs=0x5594dad98470) at tile.cpp:2576
#8  0x00007f5575c0f6db in start_thread (arg=0x7f54a32de700) at pthread_create.c:463
#9  0x00007f5574ff988f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 1 (Thread 0x7f54a3adf700 (LWP 7274)):
#0  0x00005594d9a7c531 in deserialize_feature (geoms=0x7f41181e4650, geompos_in=0x7f54a3adeeb0, z=4294967292, tx=2736259840, ty=3397699503, initial_x=0x7fff97438240, initial_y=0x7fff974381f0) at serial.cpp:275
#1  0x00005594d9a1ebb6 in next_feature (geoms=geoms@entry=0x7f41181e4650, geompos_in=geompos_in@entry=0x7f54a3adeeb0, z=z@entry=-4, tx=tx@entry=2736259840, ty=ty@entry=3397699503, initial_x=<optimized out>, initial_y=<optimized out>, original_features=<optimized out>, unclipped_features=<optimized out>, nextzoom=<optimized out>, maxzoom=<optimized out>, minzoom=<optimized out>, max_zoom_increment=<optimized out>, pass=<optimized out>, passes=<optimized out>, along=<optimized out>, alongminus=<optimized out>, buffer=<optimized out>, within=<optimized out>, first_time=<optimized out>, geomfile=<optimized out>, geompos=<optimized out>, oprogress=<optimized out>, todo=todo@entry=12849808566, fname=<optimized out>, child_shards=<optimized out>, filter=<optimized out>, stringpool=<optimized out>, pool_off=<optimized out>, layer_unmaps=<optimized out>) at tile.cpp:1318
#2  0x00005594d9a28930 in write_tile (geoms=geoms@entry=0x7f41181e4650, geompos_in=geompos_in@entry=0x7f54a3adeeb0, stringpool=<optimized out>, z=<optimized out>, tx=<optimized out>, ty=<optimized out>, detail=<optimized out>, min_detail=<optimized out>, outdb=<optimized out>, outdir=<optimized out>, buffer=<optimized out>, fname=<optimized out>, geomfile=<optimized out>, minzoom=<optimized out>, maxzoom=<optimized out>, todo=<optimized out>, along=<optimized out>, alongminus=<optimized out>, gamma=<optimized out>, child_shards=<optimized out>, pool_off=<optimized out>, initial_x=<optimized out>, initial_y=<optimized out>, running=<optimized out>, simplification=<optimized out>, layermaps=<optimized out>, layer_unmaps=<optimized out>, tiling_seg=<optimized out>, pass=<optimized out>, passes=<optimized out>, mingap=<optimized out>, minextent=<optimized out>, fraction=<optimized out>, prefilter=<optimized out>, postfilter=<optimized out>, filter=<optimized out>, arg=<optimized out>) at tile.cpp:1852
#3  0x00005594d9a2dd6a in run_thread (vargs=0x5594dad97c90) at tile.cpp:2576
#4  0x00007f5575c0f6db in start_thread (arg=0x7f54a3adf700) at pthread_create.c:463
#5  0x00007f5574ff988f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
stevage commented 4 years ago

@ericfischer Were you able to make any progress with the file I sent back in Feb? I emailed it to the address on your Github profile page.

mcfwesh commented 2 years ago

@stevage I am facing the exact problem. Did you get a solution over the past year and half now?