search

ckan / ckanext-spatial

Geospatial extension for CKAN
http://docs.ckan.org/projects/ckanext-spatial
125 stars 190 forks source link

solr-spatial-field backend fails on longitude > 180 degree harvests #307

Open benjwadams opened 1 year ago

benjwadams commented 1 year ago

I'm using solr-spatial-field as the CKAN spatial backend, along with ckanext-harvest to kick off harvests.

When some datasets with > 180 degrees of latitude are attempting to harvest, the harvester crashes due to an unhandled exception from Solr:

2023-03-01 20:58:57,773 ERROR [ckan.lib.search] Solr returned an error: Solr responded with an error (HTTP 400): [Reason: ERROR: [doc=b0b4a646f731d1fb151af2ad5441d62b] Error adding field 'spatial_geom'='POLYGON ((160 0, 257 0, 257 54, 16$
 54, 160 0))' msg=Unable to parse shape given formats "lat,lon", "x y" or as WKT because org.locationtech.spatial4j.exception.InvalidShapeException: Bad X value 257.0 is not in boundary Rect(minX=-180.0,maxX=180.0,minY=-90.0,maxY=90.0) i$
put: POLYGON ((160 0, 257 0, 257 54, 160 54, 160 0))]
bonnland commented 1 year ago

The documentation mentions that you'll have to split bboxes across the anti-meridian in order to index in Solr:

https://docs.ckan.org/projects/ckanext-spatial/en/latest/spatial-search.html?highlight=meridian#custom-indexing-logic

benjwadams commented 1 year ago

I ended up creating a fix for this, but is there any reason this can't be fixed here, or skipped, instead of crashing the harvest process?

bonnland commented 1 year ago

There is a chance it wasn't as high a priority as other issues. I am not working on CKAN development full time, but I would imagine that your solution could be useful to others, and you could make a pull request to let others benefit from your solution.

benjwadams commented 1 year ago

I'll try to contribute back, will have to make a unit test for the geometry splitting functionality.

markstuart commented 1 year ago

I'd be interested in a solution for this too @benjwadams, have you added the geometry splitting in an extension or repo where I could maybe test the code out? Or maybe you could raise a work in progress PR against this repo and add the test later on?