codice / ddf

DDF Distributed Data Framework - an open source, modular integration framework.
http://ddf.codice.org
Other
132 stars 181 forks source link

[2.26.x] Prevent AND queries containing invalid filters from running against WFS 1.1.0 sources #6736

Closed jrnorth closed 1 year ago

jrnorth commented 1 year ago

What does this PR do?

Now, the WfsFilterDelegate will return null when ANDing filters and at least one is null. This means any AND group will be dropped if it contains a feature property not understood by the source being queried. If this results in no filter for the query at all, then it will not be run against the source.

Also, note that test coverage did not actually decrease with this change. The instruction coverage was updated to reflect its true value; LenientLimit has prevented this from failing the build.

Who is reviewing it?

@derekwilhelm
@alexabird

Select relevant component teams:

@codice/core-apis @codice/ogc

Ask 2 committers to review/merge the PR and tag them here.

@glenhein
@jlcsmith

How should this be tested?

Test in downstsream

Checklist:

Notes on Review Process

Please see Notes on Review Process for further guidance on requirements for merging and abbreviated reviews.

Review Comment Legend:

jlcsmith commented 1 year ago

@jrnorth - are there potential issues here for queries that contain internal attributes like metacard-tags that don't apply to WFS sources? It seems like there are some cases where the query should return 0 results if the user issued and "AND" query where one of the attributes doesn't exist in the WFS source, but other cases where DDF is adding things like tags to the query that make sense for Solr queries but not for WFS and shouldn't cause the query to fail.

jrnorth commented 1 year ago

@jlcsmith Good question. I'm not sure if the catalog framework already handles some of that for us, like removing internal attributes for queries going to federated sources. I'll look into it.

jrnorth commented 1 year ago

@jlcsmith It's probably going to be difficult to say what attributes could possibly appear in the query. What if we were to add a configuration that lists attributes that should be dropped from the query? Do we have any sources that do something like that?

jlcsmith commented 1 year ago

We should probably handle this case downstream via config as you suggest @jrnorth

jlcsmith commented 1 year ago

build now

cxddfbot commented 1 year ago

Internal build has been started, your results will be available at build completion.

cxddfbot commented 1 year ago

Build SUCCESS See the job results in legacy Jenkins UI or in Blue Ocean UI.

clockard commented 1 year ago

@jlcsmith @jrnorth I believe that when we add internal query parameters we do so only when querying the local source because other federated systems probably wouldn't know what to do with them. So when a query hits a wfs source it shouldn't have any metacard-tags or other internal attributes in the query.

alexabird commented 1 year ago

🎉 Hero Successful 🎉