opengeospatial / geopackage

An asciidoc version of the GeoPackage specification for easier collaboration
https://www.geopackage.org
Other
264 stars 71 forks source link

Conflict between a GeoPackage SQLite Extension and libspatialite #32

Closed rouault closed 11 years ago

rouault commented 11 years ago

The fact that GeoPackage has dropped the Spatialite geometry blob will likely cause an integration nightmare for an application (e.g. GDAL/OGR) that intends supporting both GeoPackage and spatialite databases.

The reason is that both libspatialite and the GeoPackage SQLite exension (e.g. the Luciad implementation) will try to register a STMinX() (and other similar ST functions) SQLite function, but depending on the order the sqlite extensions are loaded, one will win over the other one, and will only be able to process the format of geometry blob it supports. So only spatialite or exclusively only geopackage could be handled.

How to workaround that ? Several ideas :

All the above is a bit speculative, since I did not actually try, but I do think the issue is plausible.

pdaisey commented 11 years ago

Even: The GeoPackage SWG spent weeks deciding to not use the SpatiaLite BLOB geometry format, or any existing open source or proprietary one,
so I doubt the easiest solution you propose has much of a chance of being adopted. But the integration problem is more extensive than that, since clause 3.1.1 provides a way for a BLOB geometry format other than the GeoPackage one to be registered as an extension. The SpatiaLite BLOB geometry format or any other open source or proprietary format could be a registered extension and co-exist with the GeoPackage one in the same GeoPackage file.

 In such circumstances the minimal runtime SQL (geometry) functions 

need to work for GeoPackage BLOB geometries, and at least fail gracefully on other formats. There is no statement to this effect in the present spec draft, but your comment points out the need for one.
The GeoPackage BLOB geometry format starts with some "magic" bytes and is easily identified. Other BLOB geometry formats can be identified by similar "magic" bytes if provided, or by data in rows in the gpkg_extensions table as follows if not.

 The registration of extension geometry types described in clause 

3.1.1 is required for every applicable geometry column by Req 69 with a name in the form _geometry by Req 68.

 The registration of geometry type triggers described in clause 

3.1.4 requires the minimal runtime SQL (geometry) functions in Annex D.
The registration of such triggers is required on for every applicable geometry column by Req 78 with the name "gpkg_gtype_trigger" by Req 77.

 So an implementer of a GeoPackage SQLite Extension with geometry 

functions that can handle both GeoPackage and other BLOB geometry formats is going to have to include some dispatch code to identify the type of a geometry from its 'magic' bytes if available or gpkg_extensions registration information if not to invoke the geometry function code for the applicable geometry encoding. This is not as straight forward as simply linking in other libraries. But I'd argue it isn't a nightmare, just drudgery.

 The "ST_"* functions have implementations for most every BLOB 

geometry type, so I don't think it is appropriate to change the function names in a GeoPackage.

 Regards,

Paul

On 8/12/2013 4:47 PM, Even Rouault wrote:

The fact that GeoPackage has dropped the Spatialite geometry blob will likely cause an integration nightmare for an application (e.g. GDAL/OGR) that intends supporting both GeoPackage and spatialite databases.

The reason is that both libspatialite and the GeoPackage SQLite exension (e.g. the Luciad implementation) will try to register a STMinX() (and other similar ST functions) SQLite function, but depending on the order the sqlite extensions are loaded, one will win over the other one, and will only be able to process the format of geometry blob it supports. So only spatialite or exclusively only geopackage could be handled.

How to workaround that ? Several ideas :

  • The easier solution : use the spatialite binary blob, so that any of the implementation can be used. Although I'm not sure it will always work. For example ST_Tranform() needs to query the spatial_ref_sys table to do the reprojection. The spatialite implementation requires the proj4text column in spatial_ref_sys for that, whereas an implementation of GeoPackage will likely not assume that such a column exists...
  • or make both concurrent implementation detect if the database is well the relevant one before registering their functions, i.e. a GeoPackage SQLite function would test the presence of a spatialite table (e.g. spatial_refsys with a proj4text column) and would avoid loading its function, and similarly libspatialite would test the presence of gpkg table
  • or decorate GeoPackage SQL function with a "gpkg_" prefix

All the above is a bit speculative, since I did not actually try, but I do think the issue is plausible.

— Reply to this email directly or view it on GitHub https://github.com/opengis/geopackage/issues/32.

Paul W. Daisey, Jr.
Image Matters LLC
201 Loudoun Street SW
Leesburg, VA 20175

Phone  703-669-5510
Fax    703-669-5515
Cell   301-651-7148
pauld@imagemattersllc.com  <mailto://pauld@imagemattersllc.com>

http://www.imagemattersllc.com

pepijnve commented 11 years ago

If you make sure spatialite and whatever geopackage implementation you're using are not autoloaded extensions then this problem will not present itself. Sqlite extensions work on a per connection basis. If you only load the extension you need there isn't going to be a conflict.

The only place where this could cause problems is when you attach a geopackage database in a spatialite database or vice versa.

That being said, it's on my todo list to add spatialite blob compatibility to libgpkg since we need that ourselves as well. You won't get the GEOS and PROJ integration, but you will be able to read/write spatialite files.

pepijnve commented 11 years ago

Clause 3.1.1 has been moved to clause 3.3.1, clause 3 has been renamed Other Tables, clause 2.5 has been reworded to exclude reference to other data tables by rows in the gpkg_geometry_columns table, and a footnote has been added to Table 36 in Annex D to make it clear that the geometry functions are only required to handle the GeoPackageBinary geometry encoding allowed in feature tables, and may or may not handle any extension geometry encodings allowed in other data tables, such as that defined by SpatiaLite.