Support "any" filters at top level

[JasonStoltz]

Use case: Searching for "parks" which are in "California" OR are "World Heritage Sites"

This is currently not possible. The following actions:

addFilter("states", "California", "any")
addFilter("states", "Alaska", "any")
addFilter("world_heritage_site", "true", "any")

Would result in the query:

"filters": {
    "all": [
      {
        "any": [
          { "states": "California" },
          { "states": "Alaska" }
        ]
      },
      {
        "any": [
          { "world_heritage_site": "true" }
        ]
      }
    ]
  }

Currently, the "any" parameter affects the relation of a filter to other filters on the same field, NOT the relation to other fields, which is that top level "all".

We could provide a second parameter to choose "all" or "any" at the top level:

addFilter("states", "California", "any", "any")
addFilter("states", "Alaska", "any",  "any")
addFilter("world_heritage_site", "true", "any")

"filters": {
    "any": [
      {
        "any": [
          { "states": "California" },
          { "states": "Alaska" }
        ]
      },
      {
        "any": [
          { "world_heritage_site": "true" }
        ]
      }
    ]
  }

A real world use case for this:

Like while checking package delivery availability one person might want to see if the delivery is available at Office Address OR Home Address

There could be other solutions for this, but this is my current proposal.

In the meantime, users can work around this bye programatically changing the outbound API request: https://github.com/elastic/search-ui/blob/master/ADVANCED.md#api-config

[abhishekKrHaith11]

This will also give flexibility to create filter structure like below -

"filters": {
    "all": [
      { "states": "California" },
      { "world_heritage_site": "true" }
    ],
    "any": [
      { "acres": { "from": 40000 } },
      { "square_km": { "from": 500 } }
    ],
    "none": [
      { "title": "Yosemite" }
    ]
  }

@JasonStoltz can we also find a way to use nested filters as documented in SwiftType documentation - https://swiftype.com/documentation/app-search/api/search/filters#nesting-filters That will give more flexibility with filters like below -

"filters": {
    "any": [
      {
        "all": [
          { "states": "California" },
          { "world_heritage_site": "true" }
        ]
      }
    ]
  }

[apellizzn]

What should be the default value for the top level filter ?

[JasonStoltz]

"all" would be the default at the top level.

[JasonStoltz]

@abhishekKrHaith11 The first structure you showed would not be possible, but something functionally equivalent would be.

addFilter("states", "California", "all", "all")
addFilter("world_heritage_site", "true", "all", "all")
addFilter("acres", "40000", "all", "any")
addFilter("square_km", "40000", "all", "any")
addFilter("title", "Yosemite", "all", "none")

Would give you:

"filters": {
    "all": [
      { "all":  [ { "states": "California" } ] },
      { "all":  [ { "world_heritage_site": "true" } ] }
    ],
    "any": [
      { "all":  [ { "acres": { "from": 40000 } } ] },
      { "all":  [ { "square_km": { "from": 500 } } ] }
    ],
    "none": [
       { "all":  [ { "title": "Yosemite" } ] }
    ]
  }

So in other words, we ALWAYS use a nested structure two levels deep.

[JasonStoltz]

The one thing unique about this, is there is logic that looks to see if a matching filter is already applied to that field, and if so, it adds the value to that existing filter.

For example, calling to "addFilters" on the same field with the same combination of "all" and "any":

addFilter("states", "California", "any",)
addFilter("states", "Aslaska", "any")

Would result in:

"filters": {
    "all": [
      { "any":  [ 
        { "states": "California" },
        { "states": "Alaska" } 
      ] },
    ]
  }

Just thinking out loud here, that means you wouldn't be able to apply nested "any" filters on different fields. Like you wouldn't be able to do the following, which shows two filters from DIFFERENT fields in the same any block:

"filters": {
    "all": [
      { "any":  [ 
        { "states": "California" },
        { "world_heritage_site": "true" } 
      ] },
    ]
  }

And you also wouldn't be able to do the following, which shows two filters for THE SAME field in different any blocks:

"filters": {
    "all": [
      { "any":  [ 
        { "states": "California" }
      ] },
      { "any":  [ 
        { "states": "Alaska" }
      ] }
    ]
  }

[apellizzn]

@JasonStoltz I am thinking about adding a PR for this, can we assume top level filters would only be "any", "all" and "none" ?

[JasonStoltz]

Hey @apellizzn Yes, "any", "all", and "none" would be the only valid values.

[johanarnor]

Hi! I just wanted to weigh in with some thoughts about this feature as well. I dug around in the code to check how the mapping was made from the state to the actual query, and then it hit me that it maybe would make more sense if we group the filters by field name? I.e instead of

"filters": {
  "all": [
    {
      "any": [
        { "states": "California" },
        { "states": "Alaska" }
      ]
    },
    {
      "any": [
        { "world_heritage_site": "true" }
      ]
    }
  ]
}

group by field name...

"filters": {
  "all": [
    {
      "any": [
        { "states": ["California", "Alaska"] }
      ]
    },
    {
      "any": [
        { "world_heritage_site": ["true"] }
      ]
    }
  ]
}

This would make it impossible to create an "impossible" filter - i.e. one that cannot return any hits.

"all": [
  { "states": "California" },
  { "states": "Alaska" }
]

It would however be breaking since the filter above would be treated as OR instead of AND. However, it would possibly eliminate the need of a second all/any/none parameter by treating the existing as top-level instead.

What do you think?

[JasonStoltz]

This:

{ "states": ["California", "Alaska"] }

Is functionally equivalent to:

{
  "any": [
    { "states": "California" },
    { "states": "Alaska" }
  ]
}

So if we were going to group by field name the example you provided would then become:

"filters": {
  "all": [
    {
      "states": ["California", "Alaska"]
    },
    {
      "world_heritage_site": ["true"]
    }
  ]
}

It's basically "shorthand". The problem is, there is no "shorthand" for an all query. So you'd end up writing "all" queries with the longhand syntax:

"filters": {
  "all": [
    {
      "all": [
        { "states": "California" },
        { "states":  "Alaska" }
      ]
    },
    {
      "world_heritage_site": ["true"]
    }
  ]
}

I think it's confusing and harder to code mixing the longhand and shorthand. So I just default to longhand:

"filters": {
  "all": [
    {
      "all": [
        { "states": "California" },
        { "states":  "Alaska" }
      ]
    },
    {
      "any": [
        { "world_heritage_site": "true" }
      ]
    }
  ]
}

The way I reason about it is like this:

"filters": {
  "all": [ // The relationship of all facets to one another
    { // A Facet on a field
      "all": [ // The relationship between selected values from that facet
        // The selected values from that facet
      ]
    },
    { // A Facet on a field
      "any": [ // The relationship between selected values from that facet
        // The selected values from that facet
      ]
    },
  ]
}

The grouping of this query kind of mimics what you'd see in an actual search ui experience.

[johanarnor]

I think it's confusing and harder to code mixing the longhand and shorthand. So I just default to longhand:

"filters": {
  "all": [
    {
      "all": [
        { "states": "California" },
        { "states":  "Alaska" }
      ]
    },
    {
      "any": [
        { "world_heritage_site": "true" }
      ]
    }
  ]
}

Yeah, I agree that it's confusing to mix the two syntaxes. But I also tried to think of a single use case for an all query within a single field. The following is quite weird and doesn't seem to hava a real world use case?

"all": [
{ "states": "California" },
{ "states":  "Alaska" }
]

[JasonStoltz]

The real world use case would be any Facet where you want to select multiple values to filter on that are all true.

Like if you are trying to find a hotel room to stay in and you have a Facet for amenities.

"all": [
  { "ammenity": "Non-smoking" },
  { "ammenity": "Pet-friendly" } 
]

You don't want to find a hotel that is pet friendly or non smoking, you want both of those things to be true.

[johanarnor]

Ah, now I see. That would require ammenity to be stored as an array I suppose?

[JasonStoltz]

Yes.

On Tue, Oct 29, 2019, 8:39 AM Johan Arnör notifications@github.com wrote:

Ah, now I see. That would require ammenity to be stored as an array I suppose?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/elastic/search-ui/issues/411?email_source=notifications&email_token=AAK4QEZ63CELPCFHUNJLX4TQRAVGTA5CNFSM4JAPVNN2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECQKOKQ#issuecomment-547399466, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAK4QE2FFJ7V4RYVGDGJK3LQRAVGTANCNFSM4JAPVNNQ .

[JasonStoltz]

Noting here that this request was also made in https://github.com/elastic/search-ui/issues/411.

[DickyKwok]

Can we use this feature now? addFilter("states", "California", "any", "any") 😣😣😣

[DickyKwok]

As mentioned above, How can we work on it by programatically changing the outbound API request to support or fileter (https://github.com/elastic/search-ui/blob/master/ADVANCED.md#api-config)? I am desperate for this feature, can somebody save my world?

[JasonStoltz]

Hey! Would love to help you out, not sure how soon we can get to this, unfortunately.

[MattLJoslin]

Hey, this seems to have gone stale but this is a pretty major blocker for using this package. Otherwise this has been a great solution but this seems like a big missing piece for implementing things like role based search etc. Anything we can do to help move this along? :)