WKBParser w/spherical factory parses some geometries successfully, returns nil for others

[wrkrb33]

Using RGeo 0.4.0. The ewkb geometries I'm working with are polygons pulled from a postgis database. I am using them in all sorts of contexts and know them to be valid. However, parsing fails on some percentage of them (10-20%?). I haven't been able to identify any common characteristic among the polygons that fail to parse.

> factory = RGeo::Geographic.spherical_factory(:srid=>4326)
=> #<RGeo::Geographic::Factory:0x00000002a2c1a8.....
> parser = RGeo::WKRep::WKBParser.new(factory, :support_ewkb=>true)
 => #<RGeo::WKRep::WKBParser:0x000000029f0180.....
> ewkb1 = "0103000020E6100000010000000F000000216F17D9E40958C05A5095CF557D4640C4DF3FD9E40958C0AB65939B557D4640822F2F10F50958C0FC33FDDE557D4640C43D3EB6F60958C064A48FE0557D4640A71EF2C3480A58C0C2714B2F567D4640AD35C4779C0A58C0CBDC8F7F567D46401BF1CD2B9B0A58C00A2E324ACF7D4640A62DCDDF990A58C04C64D114487E4640BE647BB85B0A58C097D32477477E46400B6D73C8450A58C0192E363F477E464090325922260A58C091E54EEE467E4640E57311B1F10958C01BE6BD67467E4640520EAE60F30958C048AA5E23CE7D46402369D379E40958C0F3BB7579CE7D4640216F17D9E40958C05A5095CF557D4640"
> ewkb2 = "0103000020E6100000010000001800000041F9969B7D5B57C0616D371F0AB44640CD9A9A9A7D5B57C0140A329D08B446408C430A72CF5B57C0F754B66009B4464062E86360D05B57C07855EC6209B446405A83FA80D05B57C070F5396309B44640984A28D60B5C57C02BA4CDEF09B446406DBD4165225C57C01BDCFF240AB446401FCDD269225C57C0F6300A250AB44640A3394148235C57C0DD7F16270AB4464008B57167235C57C0D65760270AB446405E37C068225C57C0BDDB728154B446400B2DAE68225C57C017C5C58654B446402624AD68225C57C06AC31C8754B4464070B225D9215C57C02235EC6C7EB446404DDAC3E0CF5B57C02F9A8C077FB446400B8ABA94CD5B57C0B558DB0B7FB44640C6D2ECAAC05B57C0355A07247FB4464007DCEFF29C5B57C0DBDEA5667FB44640EBB61099985B57C06550BC6E7FB44640768C0422865B57C008E804917FB446407B2EC626805B57C016671B9C7FB4464058AB5DE87D5B57C0470F5C977FB4464099ACC3A57D5B57C0AA47E8AF19B4464041F9969B7D5B57C0616D371F0AB44640"
> parser.parse ewkb1
 => #<RGeo::Geographic::SphericalPolygonImpl:0x18580bc "POLYGON ((-96.15459277428137 44.97918123999325, -96.15459281194404 44.97917504021765, -96.15558247192351 44.97918307650977, -96.1556830985856 44.97918326391007, -96.1606912483645 44.97919264968688, -96.16580003892041 44.97920221828971, -96.16572089301592 44.98288848352611, -96.16564173733659 44.98657474731644, -96.16184818315648 44.98655595110102, -96.16050921699586 44.98654928346168, -96.15857752524084 44.98653963901882, -96.15537668899886 44.986523597454116, -96.15547959325457 44.982853337500785, -96.15457006116462 44.982863600242034, -96.15459277428137 44.97918123999325))">
> parser.parse ewkb2
 => nil

[pankas87]

I'm making a guess here...

Are there any self intersecting polygons in your data?

[wrkrb33]

I'm pretty sure that there aren't - I don't believe postgis would accept them in that case. I should probably note though that they are stored in UTM Zone 15N (EPSG:26915), and I am reprojecting them in postgis to Geographic (EPSG:4326) before attempting to parse them. Not sure how that could make a difference, though...

[pankas87]

PostGIS accepts them, but I've found that when GEOS library is not correctly linked to RGeo, it cannot parse self-intersecting polygons.

[wrkrb33]

As regards GEOS, I think I'm correctly set up:

 > RGeo::Geos.supported?
 => true 

As regards validity of the polygons, the example that fails to parse above returns 't' for the below in postgis, which I believe indicates that it is not self-intersecting:

select st_isvalid('0103000020E6100000010000001800000041F9969B7D5B57C0616D371F0AB44640CD9A9A9A7D5B57C0140A329D08B446408C430A72CF5B57C0F754B66009B4464062E86360D05B57C07855EC6209B446405A83FA80D05B57C070F5396309B44640984A28D60B5C57C02BA4CDEF09B446406DBD4165225C57C01BDCFF240AB446401FCDD269225C57C0F6300A250AB44640A3394148235C57C0DD7F16270AB4464008B57167235C57C0D65760270AB446405E37C068225C57C0BDDB728154B446400B2DAE68225C57C017C5C58654B446402624AD68225C57C06AC31C8754B4464070B225D9215C57C02235EC6C7EB446404DDAC3E0CF5B57C02F9A8C077FB446400B8ABA94CD5B57C0B558DB0B7FB44640C6D2ECAAC05B57C0355A07247FB4464007DCEFF29C5B57C0DBDEA5667FB44640EBB61099985B57C06550BC6E7FB44640768C0422865B57C008E804917FB446407B2EC626805B57C016671B9C7FB4464058AB5DE87D5B57C0470F5C977FB4464099ACC3A57D5B57C0AA47E8AF19B4464041F9969B7D5B57C0616D371F0AB44640')

[nearapogee]

@wrkrb33 Where you able to figure out what the issue was? I believe I found the issue 2 days ago and reported it here: https://github.com/rgeo/activerecord-postgis-adapter/issues/202

I'd love to help get some movement here!

[wrkrb33]

@nearapogee No, I have not been able to get to the bottom of this. It is a bit of a show-stopper...

[Mr0grog]

I'm seeing a similar issue in a codebase I've recently started working on and was able to narrow it down (in at least one of the cases) to a polygon that has a few consecutive duplicate points.

I finally found this by calling is_simple? on the the rings of the polygon and noticing that it threw an exception down in ArcXYZ#degenerate?—when the start and end points of an arc are the same, axis returns nil.

I'm not sure if this exception is what is breaking parsing, but the consecutive duplicate points are definitely the problem in the data. Fixing them allowed parsing to work fine. Not sure if this is what @wrkrb33 was encountering, but it could be.

(Relatedly, it would be nice this error was handled with a meaningful exception during parsing, e.g. "consecutive identical points" or something. That would have made finding this issue much more obvious.)

[rossettistone]

I don't have much to add here except for a "me too", but I can confirm that: • All of my records (succeeding and failing) pass the st_isvalid() test. • I'm seeing this behavior consistently after saving certain WKT MULTIPOLYGON records, but not on others.

I'm working with activerecord-postgis-adapter, and got here via @nearapogee's issue above. Condensed example from my app:

> region_text = "MULTIPOLYGON (...)"
> ActiveRecord::Base.connection.execute("UPDATE districts SET district_area = ST_Transform(ST_GeomFromText('#{region_text}',102645),4326) WHERE id = #{district.id}")
> District.find(district.id).district_area
=> nil

[sosso]

I've run into this locally as well; in lib/rgeo/types.rb, adding

nfactory_.line_string(obj_.points.chunk{|x| x}.map(&:first).map { |p_| cast(p_, nfactory_, opts_) })

has fixed our problems. The key bit is the chunk and subsequent map to remove equivalent points. I've added this for the ntpye == LinearRing and LineString blocks.

[keithdoggett]

Same as #212. The geographic factory sometimes invalidates linear rings if the points are close due to floating point precision errors.