search

ckan / ckanext-spatial

Geospatial extension for CKAN
http://docs.ckan.org/projects/ckanext-spatial
125 stars 190 forks source link

Solr queries not working due to local parameters not being allowed since CKAN 2.10.4/2.9.11 #328

Open mjanez opened 3 months ago

mjanez commented 3 months ago

Environment

Problem:

solr-bbox log:

  2024-04-12 13:26:32,813 INFO  [ckan.lib.search.query] SearchError query: {'facet.field': ['theme', 'groups', 'theme_es', 'dcat_type', 'publisher_name', 'publisher_type', 'spatial_uri', 'owner_org', 'res_format', 'frequency', 'tags', 'tag_uri', 'conforms_to'], 'fq': ['  -dataset_type:harvest', '{!frange incl=false l=0 u=1}div(mul(mul(max(0,sub(min(7.910156250000001,maxx),max(-29.355468750000004,minx))),max(0,sub(min(45.91485503041036,maxy),max(23.193387206172922,miny)))),2),add(846.7296993875983,mul(sub(maxy,miny),sub(maxx,minx))))', '+site_id:"default"', '+state:active'], 'q': '*:*', 'rows': 21, 'sort': 'score desc, metadata_modified desc', 'start': 0, 'df': 'text', 'fq_list': ['{!frange incl=false l=0 u=1}div(mul(mul(max(0,sub(min(7.910156250000001,maxx),max(-29.355468750000004,minx))),max(0,sub(min(45.91485503041036,maxy),max(23.193387206172922,miny)))),2),add(846.7296993875983,mul(sub(maxy,miny),sub(maxx,minx))))'], 'bf': 'div(mul(mul(max(0,sub(min(7.910156250000001,maxx),max(-29.355468750000004,minx))),max(0,sub(min(45.91485503041036,maxy),max(23.193387206172922,miny)))),2),add(846.7296993875983,mul(sub(maxy,miny),sub(maxx,minx))))', 'defType': 'edismax', 'fl': 'id validated_data_dict', 'facet': 'true', 'facet.limit': '50', 'facet.mincount': 1, 'wt': 'json', 'tie': '0.1', 'mm': '2<-1 5<80%', 'qf': 'name^4 title^4 tags^2 groups^2 text', 'q.op': 'AND'}
  2024-04-12 13:26:32,813 ERROR [ckan.views.dataset] Dataset search error: ("Local parameters are not supported in param 'fq'. value: '{!frange incl=false l=0 u=1}div(mul(mul(max(0,sub(min(7.910156250000001,maxx),max(-29.355468750000004,minx))),max(0,sub(min(45.91485503041036,maxy),max(23.193387206172922,miny)))),2),add(846.7296993875983,mul(sub(maxy,miny),sub(maxx,minx))))'",)

solr-spatial-field log:

  2024-04-12 13:05:20,561 INFO  [ckan.lib.search.query] SearchError query: {'facet.field': ['theme', 'groups', 'theme_es', 'dcat_type', 'publisher_name', 'publisher_type', 'spatial_uri', 'owner_org', 'res_format', 'frequency', 'tags', 'tag_uri', 'conforms_to'], 'fq': ['  -dataset_type:harvest', '{!field f=spatial_geom}Intersects(ENVELOPE(-29.355468750000004, 7.910156250000001, 45.91485503041036, 23.193387206172922))', '+site_id:"default"', '+state:active'], 'q': '*:*', 'rows': 21, 'sort': 'score desc, metadata_modified desc', 'start': 0, 'df': 'text', 'fq_list': ['{!field f=spatial_geom}Intersects(ENVELOPE(-29.355468750000004, 7.910156250000001, 45.91485503041036, 23.193387206172922))'], 'fl': 'id validated_data_dict', 'facet': 'true', 'facet.limit': '50', 'facet.mincount': 1, 'wt': 'json', 'q.op': 'AND'}
  2024-04-12 13:05:20,561 ERROR [ckan.views.dataset] Dataset search error: ("Local parameters are not supported in param 'fq'. value: '{!field f=spatial_geom}Intersects(ENVELOPE(-29.355468750000004, 7.910156250000001, 45.91485503041036, 23.193387206172922))'",)

image

Reference the solution:

CKAN pull request https://github.com/ckan/ckan/pull/8053 introduces a new configuration option ckan.search.solr_allowed_query_parsers which allows to specify whitelisted local parameters. If the search parameters are not included, this will cause some problems in ckanext-spatial like {!frange}:

https://github.com/ckan/ckanext-spatial/blob/938308469892e4bcf7389cb4adee5ccdd5a0ccca/ckanext/spatial/search/__init__.py#L123-L124

or {!field}:

https://github.com/ckan/ckanext-spatial/blob/938308469892e4bcf7389cb4adee5ccdd5a0ccca/ckanext/spatial/search/__init__.py#L202-L211

Proposed fix:

Updated the doc to add params to the ckan.search.solr_allowed_query_parsers list in the ckan.ini configuration file when setting up spatial search.

Setup
-----

To enable the spatial search you need to add the ``spatial_query`` plugin to
your ini file. This plugin in turn requires the ``spatial_metadata`` plugin, eg::

  ckan.plugins = ... spatial_metadata spatial_query

To define which backend to use for the spatial search use the following
configuration option (see `Choosing a backend for the spatial search`_)::

  ckanext.spatial.search_backend = solr-bbox

For CKAN >2.10.4 and >2.9.11 to avoid errors in the spatial search, use the following configuration elements. For `solr-bbox` use `frange`, for `solr-spatial-field` use `field`::

  ckan.search.solr_allowed_query_parsers = ... frange | field

image