Support more complex queries in collections

[laurenwalker]

I realized today that we need to have an operator between filters so that filters can be ORed together. Project owners may want to create a collection based on regular query filters and the isPartOf filter.

[laurenwalker]

After thinking about this, I think the correct way to go would be to move the FIlterGroupType from the Project schema into the Collection schema, under the definition node. This means FilterGroups would be specified in the definition instead of individual Filters. This gives the user more control on how the query is constructed, by being able to group filters together.

However, this is a pretty big change that would require a hefty refactor of the MetacatUI code that constructs Filter models and collections. For that reason, I suggest we punt this issue until we have demand for more detailed queries.

Since the immediate issue is about isPartOf and id filters, I suggest we build logic into the MetacatUI code that always ORs those filters into the query.

[laurenwalker]

This feature has been requested twice recently by ESS-DIVE and Arctic Data Center team members who want to use "OR" operators between filters in the collection.

We should include plans to support this in a new version of the collection schema in our Milestone schedule.

[robyngit]

In addition to including an operator between filters in the collection definition, we would like to:

• have an option to exclude (rather than include) datasets from the query results
• allow nested groups of filters, where each group has its own operator and exclude option

We also decided that nesting of filters should theoretically be unlimited (though we will probably only allow two levels in the UI).

For example, a complex definition might look something like this:

<definition>
    <operator>OR</operator>
    <exclude>FALSE</exclude>
    <filter>....</filter>
    <filter>....</filter>
    <filter>....</filter>
    <filterGroup>
        <operator>AND</operator>
        <exclude>TRUE</exclude>
        <filter>....</filter>
        <filter>....</filter>
        <filter>....</filter>
        <filterGroup>
            <operator>OR</operator>
            <exclude>FALSE</exclude>
            <filter>....</filter>
            <filter>....</filter>
        </filterGroup>
    </filterGroup>
</definition>