search

ckan / ckanext-spatial

Geospatial extension for CKAN
http://docs.ckan.org/projects/ckanext-spatial
126 stars 193 forks source link

Searching on widget fails #218

Open filipporemonato opened 5 years ago

filipporemonato commented 5 years ago

We want to include this great extension into our website, but searching with a bounding box on the widget returns no results even if the search area completely contains the dataset extent area.

This is an example of a search over all of Norway returning no results even though a dataset lies just outside the coast:

dataset extent

search_results

Following the docs, we have selected the solr backend search by writing ckanext.spatial.search_backend = solr in ckan's configuration file (called production.ini) and updated the Solr schema with

<field name="bbox_area" type="float" indexed="true" stored="true" />
<field name="maxx" type="float" indexed="true" stored="true" />
<field name="maxy" type="float" indexed="true" stored="true" />
<field name="minx" type="float" indexed="true" stored="true" />
<field name="miny" type="float" indexed="true" stored="true" />

When rebuilding the indexing with ckan-paster --plugin=ckan search-index rebuild --config=/etc/ckan/production.ini the debug messages report

DEBUG [ckanext.spatial.plugin] Setting up the spatial model
2019-04-09 09:05:12,801 DEBUG [ckanext.spatial.model.package_extent] Spatial tables defined in memory
2019-04-09 09:05:12,809 DEBUG [ckanext.spatial.model.package_extent] Spatial tables already exist
2019-04-09 09:05:13,098 DEBUG [ckanext.spatial.plugin] Setting up the spatial model
2019-04-09 09:05:13,105 DEBUG [ckanext.spatial.model.package_extent] Spatial tables already exist

Any help would be appreciated.

RichFrazier commented 5 years ago

Having a similar issue on https://data.amerigeoss.org but get an error:

Error There was an error while searching. Please try again.

Any help would be appreciated.

abdelrahman146 commented 5 years ago

I'm having the same problem. please help!

etj commented 5 years ago

Same here.

Here a snapshot of the search page: image

Here the URL that includes the spatial filter:

http://.../dataset?q=&sort=score+desc%2C+metadata_modified+desc&ext_bbox=-121.28906250000001%2C-30.751277776257812%2C2.4609375%2C49.38237278700955&ext_prev_extent=-161.71874999999997%2C-50.28933925329178%2C42.890625%2C63.23362741232569

Here the Solr log:

2019-10-22 11:47:37.066 INFO  (qtp1791930789-16) [   x:ckan] o.a.s.c.S.Request [ckan]  webapp=/solr path=/select params={mm=2<-1+5<80%25&facet.field=my_owner_org&facet.field=my_trial&facet.field=my_platform&facet.field=my_sensor&facet.field=my_experiment&facet.field=my_classification&facet.field=my_dimension&facet.field=tags&facet.field=res_format&facet.field=license_id&bf=div(mul(mul(max(0,sub(min(2.4609375,maxx),max(-121.2890625,minx))),max(0,sub(min(49.382372787,maxy),max(-30.7512777763,miny)))),2),add(9916.5392572,mul(sub(maxy,miny),sub(maxx,minx))))&fl=id+validated_data_dict&start=0&sort=score+desc,+metadata_modified+desc&fq=+-dataset_type:harvest&fq={!frange+incl%3Dfalse+l%3D0+u%3D1}div(mul(mul(max(0,sub(min(2.4609375,maxx),max(-121.2890625,minx))),max(0,sub(min(49.382372787,maxy),max(-30.7512777763,miny)))),2),add(9916.5392572,mul(sub(maxy,miny),sub(maxx,minx))))&fq=%2Bsite_id:"default"&fq=%2Bstate:active&rows=21&fq_list={!frange+incl%3Dfalse+l%3D0+u%3D1}div(mul(mul(max(0,sub(min(2.4609375,maxx),max(-121.2890625,minx))),max(0,sub(min(49.382372787,maxy),max(-30.7512777763,miny)))),2),add(9916.5392572,mul(sub(maxy,miny),sub(maxx,minx))))&facet.limit=50&q=*:*&tie=0.1&defType=edismax&qf=name^4+title^4+tags^2+groups^2+text&facet.mincount=1&wt=json&facet=true} status=500 QTime=3
2019-10-22 11:47:37.067 ERROR (qtp1791930789-16) [   x:ckan] o.a.s.s.HttpSolrCall null:java.lang.UnsupportedOperationException
    at org.apache.lucene.queries.function.FunctionValues.floatVal(FunctionValues.java:44)
    at org.apache.lucene.queries.function.valuesource.MinFloatFunction.func(MinFloatFunction.java:42)
    at org.apache.lucene.queries.function.valuesource.MultiFloatFunction$1.floatVal(MultiFloatFunction.java:82)
    at org.apache.solr.search.ValueSourceParser$18$1.func(ValueSourceParser.java:298)
    at org.apache.lucene.queries.function.valuesource.DualFloatFunction$1.floatVal(DualFloatFunction.java:60)
    at org.apache.lucene.queries.function.valuesource.MaxFloatFunction.func(MaxFloatFunction.java:42)
    at org.apache.lucene.queries.function.valuesource.MultiFloatFunction$1.floatVal(MultiFloatFunction.java:82)
    at org.apache.lucene.queries.function.valuesource.ProductFloatFunction.func(ProductFloatFunction.java:39)
    at org.apache.lucene.queries.function.valuesource.MultiFloatFunction$1.floatVal(MultiFloatFunction.java:82)
    at org.apache.lucene.queries.function.valuesource.ProductFloatFunction.func(ProductFloatFunction.java:39)
    at org.apache.lucene.queries.function.valuesource.MultiFloatFunction$1.floatVal(MultiFloatFunction.java:82)
    at org.apache.lucene.queries.function.valuesource.DivFloatFunction.func(DivFloatFunction.java:40)
    at org.apache.lucene.queries.function.valuesource.DualFloatFunction$1.floatVal(DualFloatFunction.java:60)
    at org.apache.lucene.queries.function.FunctionValues$5.matches(FunctionValues.java:203)
    at org.apache.lucene.queries.function.ValueSourceScorer$1.matches(ValueSourceScorer.java:53)
    at org.apache.lucene.search.TwoPhaseIterator$TwoPhaseIteratorAsDocIdSetIterator.doNext(TwoPhaseIterator.java:89)
    at org.apache.lucene.search.TwoPhaseIterator$TwoPhaseIteratorAsDocIdSetIterator.nextDoc(TwoPhaseIterator.java:77)
    at org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.java:244)
    at org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:197)
    at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:39)
    at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:669)
    at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:473)
    at org.apache.solr.search.DocSetUtil.createDocSetGeneric(DocSetUtil.java:152)
    at org.apache.solr.search.DocSetUtil.createDocSet(DocSetUtil.java:141)
    at org.apache.solr.search.SolrIndexSearcher.getDocSetNC(SolrIndexSearcher.java:1603)
    at org.apache.solr.search.SolrIndexSearcher.getPositiveDocSet(SolrIndexSearcher.java:1262)
    at org.apache.solr.search.SolrIndexSearcher.getProcessedFilter(SolrIndexSearcher.java:1451)
    at org.apache.solr.search.SolrIndexSearcher.getDocListAndSetNC(SolrIndexSearcher.java:2066)
    at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1840)
    at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:609)
    at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:547)
    at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:295)
    at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:173)
    at org.apache.solr.core.SolrCore.execute(SolrCore.java:2440)
    at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:723)
    at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:529)
    at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:347)
    at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:298)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1691)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
    at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
    at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
    at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
    at org.eclipse.jetty.server.Server.handle(Server.java:534)
    at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
    at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
    at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
    at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
    at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
    at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
    at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
    at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
    at java.lang.Thread.run(Thread.java:748)

Using: - CKAN 2.8.3 - ckanext-spatial master [f9379473f25c53fcb8ec9a376fb59608c3d7013c] - SOLR 6.5.1

EDIT: a custom script removed the spatial fields from the Solr schema configuration, causing the reported error.

alpha2303 commented 2 years ago

I was facing the same issue as above comment in a project of mine. In my case, I found out that it was because of this tag in solrconfig.xml: `

true managed-schema

This overrode the 'schema.xml' with the 'managed-schema' file available in the directory, which did not contain the geospatial fields. I replaced the above tag with:

` and the rebuilt the search-index in CKAN, which fixed the issue. Hope this helps anyone else who might be facing this issue.

ChristianF88 commented 1 year ago

I am having the same issue. Unfortunately the workaround by @alpha2303 doesn't work for me as I cam not able to rebuild the search index (ckan.lib.search.common.SearchIndexError: Solr responded with an error (HTTP 500): [Reason: SolrCore 'ckan' is not available due to init failure: Could not load conf for core ckan: Error loading schema resource schema.xml] ). I am using ckan 2.9.8 with solr 8.11.2.

ChristianF88 commented 1 year ago

It's fixed I added these fields (https://docs.ckan.org/projects/ckanext-spatial/en/latest/spatial-search.html#choosing-a-backend-for-the-spatial-search) in the wrong file before.

<fields>
    <!-- ... -->
    <field name="minx" type="float" indexed="true" stored="true" />
    <field name="maxx" type="float" indexed="true" stored="true" />
    <field name="miny" type="float" indexed="true" stored="true" />
    <field name="maxy" type="float" indexed="true" stored="true" />
</fields>

Now I added the fields to /var/solr/data/ckan/conf/managed-schema, rebuild the search indexes (ckan search-index rebuild) and restarted ckan. The error is gone.

ChristianF88 commented 1 year ago

Th error is gone yay! But unfortunately the search returns no results, although it should. This seems to be the exact problem @filipporemonato opened this issue with. And it might be related to #224 and #287.

ChristianF88 commented 1 year ago

I have also tried using the pre-configured Solr Docker images for CKAN with solr-spatial. Same issue. Spatial search via bbox doesnt return anything.

By the way, there's a discrepancy between the instructions on how to modify the solr-schema between the ckanext-spatial and what is implemented in the pre-configured Solr Docker images.

Documentation adds:

<field name="minx" type="float" indexed="true" stored="true" />
<field name="maxx" type="float" indexed="true" stored="true" />
<field name="miny" type="float" indexed="true" stored="true" />
<field name="maxy" type="float" indexed="true" stored="true" />

Added in Docker file:

<field name="bbox_area" type="float" indexed="true" stored="true" />
<field name="maxx" type="float" indexed="true" stored="true" />
<field name="maxy" type="float" indexed="true" stored="true" />
<field name="minx" type="float" indexed="true" stored="true" />
<field name="miny" type="float" indexed="true" stored="true" />
ChristianF88 commented 1 year ago

@amercader - do you have an idea how to fix it or troubleshoot this issue? Cheers

amercader commented 1 year ago

@ChristianF88 for the spatial search to work you need to check two things:

  1. That Solr is properly configured with the custom fields needed. This is already done if you are using one of the -spatial Docker images.
  2. That geometries are properly indexed. A dataset needs to have a geometry attached in order to get spatially indexed in Solr. The most common way to do that is to add a custom field or extra named spatial that contains a valid GeoJSON geometry. ckanext-spatial will take care of indexing these contents on the relevant Solr fields. Alternatively you can use your own indexing logic in a custom before_dataset_index() hook to set the Solr fields manually but I'll recommend using the spatial field until you can troubleshoot the issue.

Things to try:

  1. If you are using your own Solr install and can access the admin UI, go to http://localhost:8983/solr/#/ckan/schema (replace the host to match your setup) and look for the needed fields (minx, maxx etc if using solr-bbox or spatial_geom if using solr-spatial-field)
  2. Create a dataset with a valid GeoJSON polygon in a spatial extra. Check in Solr if the field was indexed. You can see the indexed fields for a dataset using the Solr UI (http://127.0.0.1:8990/solr/#/ckan/query?q=:) or the ckan search-index show <dataset-id> command. There should be an extras_spatial field shown.
  3. Make sure there is no other plugin that implements IPackageController.before_dataset_index() that might be messing with the dict sent to Solr (ie turn off other plugins)

Hope this helps

ChristianF88 commented 1 year ago

@amercader - thank you for you input.

Using my Solr source install I can confirm that the schema and indexed test packages within Solr contain the fields required by solr-bbox:

{
  "responseHeader":{
    "status":0,
    "QTime":1,
    "params":{
      "q":"id:\"94f8749a-86ed-4677-afd1-6056ad67a0be\"",
      "indent":"true"}},
  "response":{"numFound":1,"start":0,"numFoundExact":true,"docs":[
      {
        "id":"94f8749a-86ed-4677-afd1-6056ad67a0be",
        "name":"test_package",
        "name_ngram":"test_package",
        "title":"Test_Package",
        "title_ngram":"Test_Package",
        "author":"[\"ckadm\"]",
        "maintainer":"ckadm",
        "notes":"some_note",
        "metadata_created":"2023-07-04T12:30:50.073Z",
        "metadata_modified":"2023-07-04T13:38:52.307Z",
        "state":"active",
        "organization":"test_organization",
        "validated_data_dict":"{\"author\": [\"ckadm\"], \"author_email\": null, \"creator_user_id\": \"f3cab305-ccab-4bfa-a64e-0dbeb0a4e267\", \"geographic_name\": [], \"has_part\": \"\", \"id\": \"94f8749a-86ed-4677-afd1-6056ad67a0be\", \"id_external\": \"\", \"is_part_of\": \"\", \"isopen\": false, \"license_id\": null, \"license_title\": null, \"maintainer\": \"ckadm\", \"maintainer_email\": null, \"metadata_created\": \"2023-07-04T12:30:50.073651\", \"metadata_modified\": \"2023-07-04T13:38:52.307870\", \"name\": \"test_package\", \"notes\": \"some_note\", \"notes-2\": \"\", \"num_resources\": 1, \"num_tags\": 1, \"organization\": {\"id\": \"226b4978-03eb-4b15-85cd-f9c449881076\", \"name\": \"test_organization\", \"title\": \"Test_Organization\", \"type\": \"organization\", \"description\": \"This is my organization.\", \"image_url\": \"https://www.techrepublic.com/wp-content/uploads/2017/03/meme05.jpg\", \"created\": \"2023-07-04T12:30:49.696158\", \"is_organization\": true, \"approval_status\": \"approved\", \"state\": \"active\"}, \"owner_org\": \"226b4978-03eb-4b15-85cd-f9c449881076\", \"private\": false, \"publicationlink\": \"\", \"review_level\": \"none\", \"reviewed_by\": \"\", \"spatial\": \"{\\\"type\\\": \\\"Point\\\", \\\"coordinates\\\": [8.609776496939471, 47.40384502816517]}\", \"state\": \"active\", \"status\": \"incomplete\", \"substances\": [], \"substances_generic\": [], \"systems\": [], \"tags_string\": \"some_tag\", \"taxa\": [], \"taxa_generic\": [], \"timerange\": [\"2014 TO 2016\"], \"title\": \"Test_Package\", \"type\": \"dataset\", \"url\": null, \"usage_contact\": \"ckadm\", \"variables\": [\"none\"], \"version\": null, \"resources\": [{\"cache_last_updated\": null, \"cache_url\": null, \"created\": \"2023-07-04T12:30:50.816697\", \"datastore_active\": false, \"datastore_contains_all_records_of_source_file\": false, \"description\": null, \"format\": \"JPEG\", \"hash\": \"\", \"id\": \"cc0a99b3-5903-4d24-9770-3c52946031db\", \"last_modified\": null, \"metadata_modified\": \"2023-07-04T12:30:50.812879\", \"mimetype\": \"image/jpeg\", \"mimetype_inner\": null, \"name\": \"test_resource\", \"package_id\": \"94f8749a-86ed-4677-afd1-6056ad67a0be\", \"position\": 0, \"resource_type\": \"Dataset\", \"restricted_level\": \"public\", \"size\": null, \"state\": \"active\", \"url\": \"https://static.demilked.com/wp-content/uploads/2021/07/60ed37b256b80-it-rage-comics-memes-reddit-60e6fee1e7dca__700.jpg\", \"url_type\": null}], \"tags\": [{\"display_name\": \"some_tag\", \"id\": \"e87204c3-d8e0-4647-9f23-f4a96ce179e1\", \"name\": \"some_tag\", \"state\": \"active\", \"vocabulary_id\": null}], \"groups\": [], \"relationships_as_subject\": [], \"relationships_as_object\": []}",
        "extras_geographic_name":"[]",
        "extras_review_level":"none",
        "extras_reviewed_by":"[]",
        "extras_status":"incomplete",
        "extras_substances":"[]",
        "extras_substances_generic":"[]",
        "extras_systems":"[]",
        "extras_tags_string":"some_tag",
        "extras_taxa":"[]",
        "extras_taxa_generic":"[]",
        "extras_timerange":"[\"[2014 TO 2016]\"]",
        "extras_usage_contact":"ckadm",
        "extras_variables":"[\"none\"]",
        "tags":["some_tag"],
        "capacity":"public",
        "res_name":["test_resource"],
        "res_format":["JPEG"],
        "res_url":["https://static.demilked.com/wp-content/uploads/2021/07/60ed37b256b80-it-rage-comics-memes-reddit-60e6fee1e7dca__700.jpg"],
        "res_type":["Dataset"],
        "entity_type":"package",
        "dataset_type":"dataset",
        "site_id":"default",
        "index_id":"da4066ef659c3df1bd1be08136c47bae",
        "minx":8.6097765,
        "miny":47.403847,
        "maxx":8.6097765,
        "maxy":47.403847,
        "_version_":"1770497436439019520",
        "indexed_ts":"2023-07-04T13:38:52.392Z",
        "data_dict":"{\"id\": \"94f8749a-86ed-4677-afd1-6056ad67a0be\", \"name\": \"test_package\", \"title\": \"Test_Package\", \"version\": null, \"url\": null, \"author\": \"[\\\"ckadm\\\"]\", \"author_email\": null, \"maintainer\": \"ckadm\", \"maintainer_email\": null, \"notes\": \"some_note\", \"license_id\": null, \"type\": \"dataset\", \"owner_org\": \"226b4978-03eb-4b15-85cd-f9c449881076\", \"creator_user_id\": \"f3cab305-ccab-4bfa-a64e-0dbeb0a4e267\", \"metadata_created\": \"2023-07-04T12:30:50.073651\", \"metadata_modified\": \"2023-07-04T13:38:52.307870\", \"private\": false, \"state\": \"active\", \"resources\": [{\"id\": \"cc0a99b3-5903-4d24-9770-3c52946031db\", \"package_id\": \"94f8749a-86ed-4677-afd1-6056ad67a0be\", \"url\": \"https://static.demilked.com/wp-content/uploads/2021/07/60ed37b256b80-it-rage-comics-memes-reddit-60e6fee1e7dca__700.jpg\", \"format\": \"JPEG\", \"description\": null, \"hash\": \"\", \"position\": 0, \"name\": \"test_resource\", \"resource_type\": \"Dataset\", \"mimetype\": \"image/jpeg\", \"mimetype_inner\": null, \"size\": null, \"created\": \"2023-07-04T12:30:50.816697\", \"last_modified\": null, \"metadata_modified\": \"2023-07-04T12:30:50.812879\", \"cache_url\": null, \"cache_last_updated\": null, \"url_type\": null, \"state\": \"active\", \"restricted_level\": \"public\", \"datastore_active\": false, \"datastore_contains_all_records_of_source_file\": false}], \"num_resources\": 1, \"tags\": [{\"id\": \"e87204c3-d8e0-4647-9f23-f4a96ce179e1\", \"name\": \"some_tag\", \"vocabulary_id\": null, \"state\": \"active\", \"display_name\": \"some_tag\"}], \"num_tags\": 1, \"extras\": [{\"id\": \"8ba28b3a-8140-4aab-8f3f-0211c813538c\", \"package_id\": \"94f8749a-86ed-4677-afd1-6056ad67a0be\", \"key\": \"geographic_name\", \"value\": \"[]\", \"state\": \"active\"}, {\"id\": \"4955ee0d-1590-4d31-9c36-8a8f9a64d1c1\", \"package_id\": \"94f8749a-86ed-4677-afd1-6056ad67a0be\", \"key\": \"has_part\", \"value\": \"\", \"state\": \"active\"}, {\"id\": \"1a952b3c-0b86-4c47-a1d6-0f930fb3e7c1\", \"package_id\": \"94f8749a-86ed-4677-afd1-6056ad67a0be\", \"key\": \"id_external\", \"value\": \"\", \"state\": \"active\"}, {\"id\": \"fd1d8f44-771c-4c52-93a3-f144667a2a49\", \"package_id\": \"94f8749a-86ed-4677-afd1-6056ad67a0be\", \"key\": \"is_part_of\", \"value\": \"\", \"state\": \"active\"}, {\"id\": \"44f58c51-9ebe-4c19-9286-8b749b93bc52\", \"package_id\": \"94f8749a-86ed-4677-afd1-6056ad67a0be\", \"key\": \"notes-2\", \"value\": \"\", \"state\": \"active\"}, {\"id\": \"88d79983-d1de-4fe5-a55f-1ec14e931fd6\", \"package_id\": \"94f8749a-86ed-4677-afd1-6056ad67a0be\", \"key\": \"publicationlink\", \"value\": \"\", \"state\": \"active\"}, {\"id\": \"4b6a09fd-86b7-4de1-9be7-a41682cded59\", \"package_id\": \"94f8749a-86ed-4677-afd1-6056ad67a0be\", \"key\": \"review_level\", \"value\": \"none\", \"state\": \"active\"}, {\"id\": \"9179426d-6bef-46ba-91eb-563b76d936ea\", \"package_id\": \"94f8749a-86ed-4677-afd1-6056ad67a0be\", \"key\": \"reviewed_by\", \"value\": \"[]\", \"state\": \"active\"}, {\"id\": \"44c05ac7-6cd4-4fb1-b8b9-efce164e89ac\", \"package_id\": \"94f8749a-86ed-4677-afd1-6056ad67a0be\", \"key\": \"spatial\", \"value\": \"{\\\"type\\\": \\\"Point\\\", \\\"coordinates\\\": [8.609776496939471, 47.40384502816517]}\", \"state\": \"active\"}, {\"id\": \"b845e7e6-597e-4661-8a70-bb4b48f2b057\", \"package_id\": \"94f8749a-86ed-4677-afd1-6056ad67a0be\", \"key\": \"status\", \"value\": \"incomplete\", \"state\": \"active\"}, {\"id\": \"460906a1-55eb-42b0-982c-23208a017d0d\", \"package_id\": \"94f8749a-86ed-4677-afd1-6056ad67a0be\", \"key\": \"substances\", \"value\": \"[]\", \"state\": \"active\"}, {\"id\": \"1b515e6b-0674-4b9a-9c0d-3509ccd3d5e0\", \"package_id\": \"94f8749a-86ed-4677-afd1-6056ad67a0be\", \"key\": \"substances_generic\", \"value\": \"[]\", \"state\": \"active\"}, {\"id\": \"eb0a33bb-5cb6-4b2a-aa1c-2e73cb87bd04\", \"package_id\": \"94f8749a-86ed-4677-afd1-6056ad67a0be\", \"key\": \"systems\", \"value\": \"[]\", \"state\": \"active\"}, {\"id\": \"a9e39b26-816a-4767-bc99-4c3ce7e193fa\", \"package_id\": \"94f8749a-86ed-4677-afd1-6056ad67a0be\", \"key\": \"tags_string\", \"value\": \"some_tag\", \"state\": \"active\"}, {\"id\": \"bce11a67-e468-497c-b336-71584909e852\", \"package_id\": \"94f8749a-86ed-4677-afd1-6056ad67a0be\", \"key\": \"taxa\", \"value\": \"[]\", \"state\": \"active\"}, {\"id\": \"79dc34d4-6439-4f98-90a0-9b55e8da303e\", \"package_id\": \"94f8749a-86ed-4677-afd1-6056ad67a0be\", \"key\": \"taxa_generic\", \"value\": \"[]\", \"state\": \"active\"}, {\"id\": \"e970f507-268f-47e6-8d2f-47de9836dcd8\", \"package_id\": \"94f8749a-86ed-4677-afd1-6056ad67a0be\", \"key\": \"timerange\", \"value\": \"[\\\"[2014 TO 2016]\\\"]\", \"state\": \"active\"}, {\"id\": \"1c404146-371a-4d87-980c-2db95fe60a8e\", \"package_id\": \"94f8749a-86ed-4677-afd1-6056ad67a0be\", \"key\": \"usage_contact\", \"value\": \"ckadm\", \"state\": \"active\"}, {\"id\": \"6d19cf68-77b4-49e3-b682-5635d15def34\", \"package_id\": \"94f8749a-86ed-4677-afd1-6056ad67a0be\", \"key\": \"variables\", \"value\": \"[\\\"none\\\"]\", \"state\": \"active\"}], \"groups\": [], \"organization\": {\"id\": \"226b4978-03eb-4b15-85cd-f9c449881076\", \"name\": \"test_organization\", \"title\": \"Test_Organization\", \"type\": \"organization\", \"description\": \"This is my organization.\", \"image_url\": \"https://www.techrepublic.com/wp-content/uploads/2017/03/meme05.jpg\", \"created\": \"2023-07-04T12:30:49.696158\", \"is_organization\": true, \"approval_status\": \"approved\", \"state\": \"active\"}, \"relationships_as_subject\": [], \"relationships_as_object\": [], \"isopen\": false, \"license_title\": null}"}]
}

So I assume of the things to try number 1 and 2 function as intended.

Running grep to see if before_dataset_index is implemented in any other plugins shows that only ckanext-spatial is implementing it.

(default) root@bookworm:/usr/lib/ckan/default/src# grep -R before_dataset_index ckanext-*
ckanext-spatial/doc/spatial-search.rst:In this case you need to implement the ``before_dataset_index()`` method of the `IPackageController <https://docs.ckan.org/en/latest/extensions/plugin-interfaces.html#ckan.plugins.interfaces.IPackageController.before_dataset_index>`_ interface::
ckanext-spatial/doc/spatial-search.rst:* Check the default implementation of ``before_dataset_index()`` in `ckanext/spatial/plugins/__init__.py <https://github.com/ckan/ckanext-spatial/blob/master/ckanext/spatial/plugin/__init__.py>`_ for extra useful checks and validations.
ckanext-spatial/ckanext/spatial/plugin/__init__.py:        return self.before_dataset_index(pkg_dict)
ckanext-spatial/ckanext/spatial/plugin/__init__.py:    def before_dataset_index(self, pkg_dict):
grep: ckanext-spatial/ckanext/spatial/plugin/__pycache__/__init__.cpython-39.pyc: binary file matches
ckanext-spatial/ckanext/spatial/tests/test_plugin/plugin.py:        return self.before_dataset_index(pkg_dict)
ckanext-spatial/ckanext/spatial/tests/test_plugin/plugin.py:    def before_dataset_index(self, pkg_dict):

Which confirms that also the 3rd point to try cannot cause the issue.

Are there any pointers from your side regarding the config?

Currently I am loading these plugins: 

ckan.plugins = ... spatial_metadata spatial_query resource_proxy geo_view  ...

and am applying these other settings:

ckan.views.default_views = image_view text_view recline_view pdf_view webpage_view geo_view

ckanext.spatial.search_backend = solr-bbox 
ckanext.geoview.ol_viewer.formats = geojson

ckanext.spatial.common_map.type = custom 
ckanext.spatial.common_map.custom.url = ...
ckanext.spatial.common_map.attribution = ...
# setting the default extend in theming extension
ckanext.eaw_theme.spatial_query_default_extent = [[45.80, 5.84], [47.84, 10.50]]
amercader commented 1 year ago

@ChristianF88 looks like you are trying to index a point ("spatial\": \"{\\\"type\\\": \\\"Point\\\", \\\"coordinates\\\": [8.609776496939471, 47.40384502816517]}). As stated in the docs, the solr-bbox backend only supports Bounding boxes and Polygons. If you want to index points you'll need to use the solr-spatial-field backend.

ChristianF88 commented 1 year ago

I added a polygon example package and it works! I think this issue can be closed. Possibly also the related issues from my previous comment .