search

wellcomecollection / catalogue-api

:crystal_ball: The API for searching the Wellcome Collection catalogue.
https://developers.wellcomecollection.org
MIT License
4 stars 0 forks source link

Move unpaired filters to query.filter #677

Closed paul-butcher closed 1 year ago

paul-butcher commented 1 year ago

With #668, I moved all the paired filters to post_filter, so that search and aggregations could operate over different sets (i.e. aggregations with a paired filter do not filter their own field when aggregating).

However, I may have gone a little too far and now all filters are post filters, regardless of pairing.

Although the end results are the same, this may make queries more expensive, as now they are operating over the whole index and filtering out unwanted results, rather than the other way round.

We should pre filter as much as possible - any unpaired filters should be pre filtered, and only those involved in the aggregations should be post filtered.

paul-butcher commented 1 year ago

This may be invalidated/obviated/included by https://github.com/wellcomecollection/catalogue-api/issues/662

When I change the way the queries are compiled, it may be best to leave them in post_filters, or there may be some other way to handle it all.