Filter on attributes, comparable to DD-ECO-API

[stefhummel]

This issue relates to #19. If indeed attributes are added to the meta data, it would be useful to be able to filter on them in the /coverages end point. The generic filter mechanism the DD-ECO-API could be used for this.

[GeriWolters]

Metadata may not be of interest of all consumers. For that reason, I've added a proposal to the DD-ECO-API to add a property that allows you to specify which properties not to return in the response. This results in smaller responses. Maybe that could be of use for the DD-API/DD-GRID-API as well?

[caspervdw]

Filtering on all attributes will introduce scaling problems. For large databases, every attribute that can be filtered upon requires an index. It depends on the database type you are using and on the actual usage (amount of writes vs amount of scans) if this is desirable or at all possible.

Therefore I would like to only make a small set of attribute filters mandatory.

[stefhummel]

For filtering we should discuss the possibilities for the comparer. The current examples use in, as defined in the DD-ECO-API filter syntax. However, we might also initially stick to a simple eq (equal). The examples in (https://github.com/DigitaleDeltaOrg/dd-grid-api/blob/master/dd-grid-api-filtering.md) then would not be: .../data?filter=realization:in:1,7,8,9,15 .../data?filter=realization:in:minimum,maximum but .../data?filter=realization:eq:1,7,8,9,15 .../data?filter=realization:eq:minimum,maximum

[GeriWolters]

The in syntax used is incorrect. It is meant to be one of the items listed in the array.

It should be: ../data?filter=realization:in:[1,7,8,9,15] .../data?filter=realization:in:[minimum,maximum]

eq always uses a single element.

[stefhummel]

Of course is the in syntax correct (and the meaning clear, hence there inclusion in the examples). I just pose the above suggestion to keep things in this first version simple and let the parser always expect and 'eq', comparable to the '=' in: .../data&quantityId=waterlevel,temperature .../data&x=1.2,3.6.4.7&y=6.1,2.6.1.7

[stefhummel]

Ah, sorry, misread @GeriWolters statement, sorry. I will adjust the examples to _in:[....,...,...] Still we should discuss the approach.

[caspervdw]

I wonder whether this style of adding "comparers" is defined as some generally accepted language? Is there a reference somewhere with a proper definition? That would ease implementation significantly.

The proposed : and [] both are all not allowed in URLs, so they would have to be encoded. The examples of @GeriWolters

.../data?filter=realization%3Aeq%3A%5B1,7,8,9,15%5D
.../data?filter=realization%3Aeq%3A%5Bminimum,maximum%5D

The styles I am used to work with mostly use periods or double underscores (__) to add some interpunction, so: realization__in=1,7,8,9,15 or Realization.In=1,7,8,9,15. These survive the url encoding process.

[stefhummel]

The idea is to take over the 'comparer approach' from the DD-API (where it is used in the query parameters locationFilter, observationtypeFilter and timeseriesFilter). The DD-API approach was copied from the DD-ECO-API, which on its turn took over the syntax from OData (see for example section 4.5. Filter System Query Option in the [OData URI Conventions] {https://www.odata.org/documentation/odata-version-2-0/uri-conventions/). For readability, the space (' ')was replace by a colon (':'). I agree that the syntax Realization.In=1,7,8,9,15 is more readable and nicer, but the '=' will cause problems because it is the expression in the query parameter filter=.expression. As decided during the last team meeting we will keep the current filtering approach in the specification, but we will explicitly state that it is not mandatory for an system to implement the filter query parameter. I have created an issue (#21) that states that we should discuss the syntax.

[gerritsj]

Zie ook OData voor filtering m.b.v. comparison operators en functions: https://docs.mendix.com/refguide/odata-query-options#4-filtering