pramsey / pgsql-ogr-fdw

PostgreSQL foreign data wrapper for OGR
MIT License
237 stars 34 forks source link

Geometry type determination (for polygon shapefiles) #164

Closed dpsspatial closed 5 years ago

dpsspatial commented 5 years ago

Not really an issue but more of a question...

How does the OGR_FDW determine the geometry type of a polygon shapefile?

We pointed a FDW at the City and County of Denver parcels table ( link ) and they were read as POLYGON, but they are really stored as MULTIPOLYGON.

I ended up doing an ALTER TABLE on the FDW and changing the geometry type to MULTI, and the FDW behaves normally. (For background, our ETL process for web-based data now gets zipped shapefiles, and unzips them in a folder filled with shapefiles which are all the source of FDW's, which we then perform an truncate and load/insert from PGIS to update the production tables... works amazingly well, especially for pesky CSV's with null bytes...)

Anyway, just wondering if the geometry type is read by a 'quick look', or something else.

THANKS!!!

pramsey commented 5 years ago

Well, the short answer is: we do whatever OGR says, @rouault maybe knows why OGR says the type of a shapefile polygon is Polygon and not MultiPolygon (maybe just legacy behaviour). The result I guess you are seeing, which is it's possible for a table to emit objects that its typmod would seem to indicate are illegal.

ogr_fdw=# \d foo
                            Foreign table "public.foo"
 Column |           Type           | Collation | Nullable | Default | FDW options 
--------+--------------------------+-----------+----------+---------+-------------
 fid    | bigint                   |           |          |         | 
 geom   | geometry(PolygonZM,4326) |           |          |         | 
 id     | double precision         |           |          |         | 
 name   | character varying        |           |          |         | 
Server: myserver
FDW options: (layer 'foo')

ogr_fdw=# select st_astext(geom) from foo;
                                                                                                                                                                                                                                                                                                                                                                  st_astext                                                                                                                                                                                                                                                                                                                                                                  
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 POLYGON ZM ((-1.19191919191919 0.118686868686869 0 0,-1.18181818181818 0.404040404040404 0 0,-0.82070707070707 0.527777777777778 0 0,-0.386363636363636 0.419191919191919 0 0,-0.26010101010101 0.138888888888889 0 0,-0.457070707070707 -0.037878787878788 0 0,-0.762626262626262 -0.313131313131313 0 0,-0.762626262626262 -0.313131313131313 0 0,-1.08585858585859 -0.232323232323233 0 0,-1.08838383838384 -0.232323232323233 0 0,-1.19191919191919 0.118686868686869 0 0))
 POLYGON ZM ((0.292929292929293 0.494949494949495 0 0,0.760101010101011 0.535353535353535 0 0,0.760101010101011 0.535353535353535 0 0,0.707070707070707 -0.154040404040404 0 0,0.090909090909091 -0.22979797979798 0 0,0.292929292929293 0.494949494949495 0 0))
 MULTIPOLYGON ZM (((-1.19191919191919 0.118686868686869 0 0,-1.18181818181818 0.404040404040404 0 0,-0.82070707070707 0.527777777777778 0 0,-0.386363636363636 0.419191919191919 0 0,-0.26010101010101 0.138888888888889 0 0,-0.457070707070707 -0.037878787878788 0 0,-0.762626262626262 -0.313131313131313 0 0,-0.762626262626262 -0.313131313131313 0 0,-1.08585858585859 -0.232323232323233 0 0,-1.08838383838384 -0.232323232323233 0 0,-1.19191919191919 0.118686868686869 0 0)),((0.292929292929293 0.494949494949495 0 0,0.760101010101011 0.535353535353535 0 0,0.760101010101011 0.535353535353535 0 0,0.707070707070707 -0.154040404040404 0 0,0.090909090909091 -0.22979797979798 0 0,0.292929292929293 0.494949494949495 0 0)))
(3 rows)

I strongly resist trying to hard code too much OGR driver-awareness into the FDW, on the theory that I should get everything I need from the OGR API. The code in question is: https://github.com/pramsey/pgsql-ogr-fdw/blob/5bad13a7d9c04db7898b0b0962df4f29f91d11a5/ogr_fdw_common.c#L268-L277

rouault commented 5 years ago

yes this is a known behaviour of the shapefile driver documented at https://gdal.org/drv_shapefile.html "The same applies for SHPT_POLYGON shapefiles, reported as layers of type wkbPolygon, but depending on the number of parts of each geometry, the actual type can be either OGRPolygon or OGRMultiPolygon. " This is due to the shapefile format itself having a single geometry type in its header for polygon & multipolygon content (same for linestring & multilinestring, but points and multipoints are separate types). A possibility (done in other clients like QGIS : https://github.com/qgis/QGIS/blob/d332292863b49b1083ee9403b13c265ab670d7a3/src/providers/ogr/qgsogrprovider.cpp#L1399) would be for OGR FDW in the case of this driver to auto-promote line->multiline and polygon->multipolygon.

pramsey commented 5 years ago

I guess it's just one driver... another brick in the special-case wall :)

dpsspatial commented 5 years ago

Really appreciate the explanation @pramsey @rouault !!! Thanks for a great tool!