geopython / pycsw

pycsw is an OGC CSW server implementation written in Python. pycsw fully implements the OpenGIS Catalogue Service Implementation Specification [Catalogue Service for the Web]. Initial development started in 2010 (more formally announced in 2011). The project is certified OGC Compliant, and is an OGC Reference Implementation. pycsw allows for the publishing and discovery of geospatial metadata via numerous APIs (CSW 2/CSW 3, OpenSearch, OAI-PMH, SRU). Existing repositories of geospatial metadata can also be exposed, providing a standards-based metadata and catalogue component of spatial data infrastructures. pycsw is Open Source, released under an MIT license, and runs on all major platforms (Windows, Linux, Mac OS X). Please read the docs at https://pycsw.org/docs for more information.
https://pycsw.org
MIT License
207 stars 155 forks source link

"Invalid Filter syntax" for all CQL_Filter requests #569

Closed mohmad-null closed 1 year ago

mohmad-null commented 6 years ago

I'm trying to do a CQL_FILTER via a GET request, but no matter what I put as the constraint, I always get back an exception "Invalid Filter syntax".

These are all against the demo pycsw deployment demo.pycsw.org

I've tried: The CQL from the GeoNetwork documentation: https://geonetwork-opensource.org/manuals/2.10.4/eng/developer/xml_services/csw_services.html#getrecords (That query is actually invalid anyway because it is missing the ElementSetName=full, but once you add it, you'll get the exception back)

Very basic examples built around this query: this: http://demo.pycsw.org/gisdata/csw?service=CSW&version=2.0.2&resultType=results&startPosition=1&maxRecords=10&typeNames=csw:Record&request=GetRecords&typeNames=csw:Record&ElementSetName=full&constraintlanguage=CQL_TEXT

with any of these constraints: &constraint=title%3Ddinosaur &constraint=dc%3Atitle%3Ddinosaur &constraint="dc:title%3Ddinosaur" &constraint=%22dc%3Atitle%3Ddinosaur%22

&CONSTRAINT=”prop1!=10” &CONSTRAINT=%22prop1!=10%22

The last one is copied and pasted straight from the CSW 2.0.2 spec.

Admittedly there's no real documentation out there for CQL ("pycsw cql constraint" finds nothing) so maybe my queries are wrong, but surely one of them should work given they appear valid, and one is in the spec. Especially after I've done the URL % encoding.

thorsten-reitz commented 4 years ago

Pretty much same here (I used queries that work against GeoNetwork), would be great to get some more helpful error messages instead of just "invalid filter syntax"...

jondoig commented 4 years ago

As mentioned here, sample requests can be found in the tests directory: for CSW 2.0.2 and CSW 3.

E.g. (CSW 2.0.2): title like '%Agriculture%'

tomkralidis commented 4 years ago

Hi all: I'm not able to reproduce. The following URLs return records (and not exceptions):

Can someone test and report back with an example that can be demonstrated against http://demo.pycsw.org ?

With regard to error messages, this is by design for security purposes. pycsw allows for server logging, which will provide deeper error messages to the administrator of the endpoint.

jokiefer commented 2 years ago

here some examples to reproduce:

http://demo.pycsw.org/gisdata/csw?service=CSW&version=2.0.2&resultType=results&startPosition=1&maxRecords=10&typeNames=csw:Record&request=GetRecords&typeNames=csw:Record&ElementSetName=full&constraintlanguage=CQL_TEXT&constraint=Type=%27dataset%27

same works fine on other csw implementations: https://www.geoportal-bw.de/geonetwork/srv/ger/csw?REQUEST=GetRecords&VERSION=2.0.2&SERVICE=CSW&typenames=gmd:MD_Metadata&resultType=hits&constraintLanguage=CQL_TEXT&Constraint=Type='dataset'&CONSTRAINT_LANGUAGE_VERSION=1.1.0

https://geoportal.brandenburg.de/csw-gdi-bb/service?REQUEST=GetRecords&VERSION=2.0.2&SERVICE=CSW&typeNames=gmd:MD_Metadata&resultType=hits&constraintLanguage=CQL_TEXT&Constraint=Type='dataset'&CONSTRAINT_LANGUAGE_VERSION=1.1.0

Update

same on xml filter:

http://demo.pycsw.org/gisdata/csw?service=CSW&version=2.0.2&resultType=results&startPosition=1&maxRecords=10&typeNames=csw:Record&request=GetRecords&typeNames=csw:Record&ElementSetName=full&constraintlanguage=CQL_TEXT&constraint=Type=%27dataset%27

https://gdk.geoportal.hessen.de/pycsw/csw.py?REQUEST=GetRecords&VERSION=2.0.2&SERVICE=CSW&typenames=gmd:MD_Metadata&constraintLanguage=FILTER&ElementSetName=full&resultType=hits&constraintLanguage=FILTER&Constraint=%3Cogc%3AFilter%20xmlns%3Aogc%3D%22http%3A%2F%2Fwww.opengis.net%2Fogc%22%3E%3Cogc%3APropertyIsEqualTo%3E%3Cogc%3APropertyName%3EType%3C%2Fogc%3APropertyName%3E%3Cogc%3ALiteral%3Edataset%3C%2Fogc%3ALiteral%3E%3C%2Fogc%3APropertyIsEqualTo%3E%3C%2Fogc%3AFilter%3E&CONSTRAINT_LANGUAGE_VERSION=1.1.0

results in:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!-- pycsw 2.3.dev0 -->
<ows:ExceptionReport xmlns:csw="http://www.opengis.net/cat/csw/2.0.2" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:dct="http://purl.org/dc/terms/" xmlns:gmd="http://www.isotc211.org/2005/gmd" xmlns:gml="http://www.opengis.net/gml" xmlns:ows="http://www.opengis.net/ows" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" language="de-DE" version="1.2.0" xsi:schemaLocation="http://www.opengis.net/ows http://schemas.opengis.net/ows/1.0.0/owsExceptionReport.xsd">
    <ows:Exception exceptionCode="InvalidParameterValue" locator="constraint">
        <ows:ExceptionText>Invalid Filter query: Exception: document not valid.
Error: Invalid PropertyName: Type.  'Type'.</ows:ExceptionText>
    </ows:Exception>
</ows:ExceptionReport>

Update 2

found my issue:

the supported queryables are with namespaces... http://demo.pycsw.org/gisdata/csw?REQUEST=GetRecords&VERSION=2.0.2&SERVICE=CSW&typenames=gmd:MD_Metadata&ElementSetName=full&resultType=hits&constraintLanguage=FILTER&CONSTRAINT_LANGUAGE_VERSION=1.0.0&outputSchema=http://www.isotc211.org/2005/gmd&Constraint=%3Cogc%3AFilter%20xmlns%3Aogc%3D%22http%3A%2F%2Fwww.opengis.net%2Fogc%22%3E%3Cogc%3APropertyIsEqualTo%3E%3Cogc%3APropertyName%3Eapiso%3AType%3C%2Fogc%3APropertyName%3E%3Cogc%3ALiteral%3Edataset%3C%2Fogc%3ALiteral%3E%3C%2Fogc%3APropertyIsEqualTo%3E%3C%2Fogc%3AFilter%3E

tomkralidis commented 2 years ago

You need to place spaces between property name, operator, and literal, and the property name needs to be bound to a namespace prefix, so:

http://demo.pycsw.org/gisdata/csw?service=CSW&version=2.0.2&resultType=results&startPosition=1&maxRecords=10&typeNames=csw:Record&request=GetRecords&typeNames=csw:Record&ElementSetName=full&constraintlanguage=CQL_TEXT&constraint=dc:type%20%3D%20%27dataset%27

http://demo.pycsw.org/gisdata/csw?service=CSW&version=2.0.2&resultType=results&startPosition=1&maxRecords=10&typeNames=csw:Record&request=GetRecords&typeNames=csw:Record&ElementSetName=full&constraintlanguage=CQL_TEXT&constraint=apiso:Type%20%3D%20%27dataset%27

pvgenuchten commented 1 year ago

A note on ogcapi-records, supported filter languages (for now) are :

https://github.com/geopython/pycsw/blob/3243c43c613d1fc951f3aacd6c00cbbeba119a6a/pycsw/ogc/api/records.py#L541-L542C13