Crashed with recent data and espg:3857

[molind]

Hello,

I used following command line to generate coasltlines data:

osmcoastline --no-index --srs=3857 --bbox-overlap=0 --max-points=30000 --output-polygons=water -v -o coastline.db coastlines.pbf

coasltlines.pbf is filtered data from recent planet dump (07 oct).

At the same time default projection works without issues.

Last lines from log:

[ 6:26] Writing out water polygons...
ERROR 1: IllegalArgumentException: Points of LinearRing do not form a closed linestring
invalid, using buffer(0)
ERROR 1: IllegalArgumentException: Points of LinearRing do not form a closed linestring
ERROR 1: IllegalArgumentException: Points of LinearRing do not form a closed linestring
invalid, using buffer(0)
ERROR 1: IllegalArgumentException: Points of LinearRing do not form a closed linestring
ERROR 1: IllegalArgumentException: Points of LinearRing do not form a closed linestring
invalid, using buffer(0)
ERROR 1: IllegalArgumentException: Points of LinearRing do not form a closed linestring
./genCoastlines: line 3: 18469 Segmentation fault      osmcoastline --no-index --srs=3857 --bbox-overlap=0 --max-points=30000 --output-polygons=water -d -v -o coastline.db coastlines.pbf

[joto]

I can not reproduce this. Can you put the exact coastlines.pbf you were using somewhere and/or get be a stack trace?

To get a stack trace compile with -g, make sure a core dump is generated ("ulimit -c unlimited" in the shell before running the program), call "gdb PROGRAMNAME core" and get me the output of "bt" command in debugger.

[molind]

There is a file link: coastlines.pbf (224.6 MB) https://mega.co.nz/#!HIwxECzJ!TUlL3nmNWf9rhVWVEAnoMh0oiO5_Crcyf4x0TR1FyVQ and i'll try to get stack trace tomorrow.

[joto]

Can't download that. Get a "update your browser" message.

[molind]

Second try: http://c.galileo-app.com/coastlines.pbf

[joto]

I still can't reproduce this. Maybe you just don't have enough memory? This run needs peak about 3.2 GB RAM.

[molind]

I have 16Gb RAM on server, so it should be enough. I'll try to get stack trace today.

[molind]

There it is:

(gdb) bt
#0  0x00000000004318d1 in CoastlinePolygons::split_bbox (this=0x7fffffffe7a0, e=..., v=0x17b5ad10) at coastline_polygons.cpp:375
#1  0x0000000000431c1d in CoastlinePolygons::output_water_polygons (this=0x7fffffffe7a0) at coastline_polygons.cpp:402
#2  0x0000000000408b7e in main (argc=11, argv=0x7fffffffea68) at osmcoastline.cpp:333

upd

(gdb) print v
$1 = (polygon_vector_t *) 0x17b5ad10
(gdb) print v->size()
$2 = 499644
(gdb) print (*it)
$3 = (OGRPolygon * const&) @0x82051138: 0x0

I have no idea why, but seems (*it) is just nullptr. that's why it crashes.

[joto]

Probably some issue with different versions of OGR or GEOS behaving slightly different. There was a place where I didn't check a return value. I have added that check. If it is the right place, you should get a warning message "Buffer(0) failed, ignoring this polygon. Output data might be invalid!" instead of the core dump and the program will move on. Please try this out.

[molind]

Yes there is lots of them

DEBUG: split_polygon(): depth=0 envelope=(-9.45682e+06, 2.25248e+06),(-8.25225e+06, 2.65474e+06) num_points=43578
[11:40] Writing out water polygons...
ERROR 1: IllegalArgumentException: Points of LinearRing do not form a closed linestring
Invalid polygon, trying buffer(0).
ERROR 1: IllegalArgumentException: Points of LinearRing do not form a closed linestring
Buffer(0) failed, ignoring this polygon. Output data might be invalid!
ERROR 1: IllegalArgumentException: Points of LinearRing do not form a closed linestring
Invalid polygon, trying buffer(0).
ERROR 1: IllegalArgumentException: Points of LinearRing do not form a closed linestring
Buffer(0) failed, ignoring this polygon. Output data might be invalid!
ERROR 1: IllegalArgumentException: Points of LinearRing do not form a closed linestring
Invalid polygon, trying buffer(0).
ERROR 1: IllegalArgumentException: Points of LinearRing do not form a closed linestring
Buffer(0) failed, ignoring this polygon. Output data might be invalid!

[joto]

Okay, so a least we found the right spot. How does the result look? Are there obvious missing pieces?

Which GDAL/OGR and GEOS versions are you using?

Can you try with smaller max-points? Try 1000.

[molind]

I used Ubuntu 13.04 when reported the bug, and now i updated to 13.10

Package: libgdal1
Version: 1.9.0-3.1ubuntu4

Package: libgeos-3.3.3
Version: 3.3.3-1.1ubuntu1

with smaller max points it still shows this error message.

Buffer(0) failed, ignoring this polygon. Output data might be invalid!
ERROR 1: IllegalArgumentException: Points of LinearRing do not form a closed linestring
Invalid polygon, trying buffer(0).
ERROR 1: IllegalArgumentException: Points of LinearRing do not form a closed linestring
Buffer(0) failed, ignoring this polygon. Output data might be invalid!
[43:39] Memory used currently: 3634 MB (Peak was: 3925 MB).
[43:39] Committing database transactions...
[43:39] All done.
[43:39] Memory used currently: 3634 MB (Peak was: 3925 MB).
There were 4 warnings.
There were 2 errors.

[joto]

Okay, I could reproduce this now with the new GDAL/GEOS libraries. Something must have changed in them that leads to this error. But with the new check in the osmcoastine code, it doesn't crash any more, so thats fine.

The reason this error appears at all seems to be related to osmcoastline's efforts to fix broken coastlines. While this mostly works, it might not always work and leave some invalid polygons. Unfortunately there isn't much we can do about that. It is still better to try instead of giving up completely. If you insist on only working with "perfect" coastlines, you have to check the return code which will only be zero if there were no problems at all. I have just changed osmcoastline to make sure at least some warnings are generated when this problem happens.

[molind]

Ok, thanks for your support and for this great project.