Add support to provide regex filters on entity queries

[fabiaserra]

Looking into porting some of our code into AYON I noticed how much more complicated the new query functions look using the new API, for example, compare this new implementation of _get_representations on the workfile template: https://github.com/ynput/ayon-core/blob/d8e5734e57c19a740f0fa2c3f4a5984754f68431/client/ayon_core/pipeline/workfile/workfile_template_builder.py#L1436 To how it was in OP before:

def _get_representations(self, placeholder):
        """Prepared query of representations based on load options.

        This function is directly connected to options defined in
        'get_load_plugin_options'.

        Note:
            This returns all representation documents from all versions of
                matching subset. To filter for last version use
                '_reduce_last_version_repre_docs'.

        Args:
            placeholder (PlaceholderItem): Item which should be populated.

        Returns:
            List[Dict[str, Any]]: Representation documents matching filters
                from placeholder data.
        """

        project_name = self.builder.project_name
        current_asset_doc = self.builder.current_asset_doc
        linked_asset_docs = self.builder.linked_asset_docs

        builder_type = placeholder.data["builder_type"]
        if builder_type == "context_asset":
            context_filters = {
                "asset": [current_asset_doc["name"]],
                "subset": [re.compile(placeholder.data["subset"])],
                "hierarchy": [re.compile(placeholder.data["hierarchy"])],
                "representation": [placeholder.data["representation"]],
                "family": [placeholder.data["family"]]
            }

        elif builder_type == "linked_asset":
            asset_regex = re.compile(placeholder.data["asset"])
            linked_asset_names = []
            for asset_doc in linked_asset_docs:
                asset_name = asset_doc["name"]
                if asset_regex.match(asset_name):
                    linked_asset_names.append(asset_name)

            context_filters = {
                "asset": linked_asset_names,
                "subset": [re.compile(placeholder.data["subset"])],
                "hierarchy": [re.compile(placeholder.data["hierarchy"])],
                "representation": [placeholder.data["representation"]],
                "family": [placeholder.data["family"]],
            }

        else:
            context_filters = {
                "asset": [re.compile(placeholder.data["asset"])],
                "subset": [re.compile(placeholder.data["subset"])],
                "hierarchy": [re.compile(placeholder.data["hierarchy"])],
                "representation": [placeholder.data["representation"]],
                "family": [placeholder.data["family"]]
            }

        return list(get_representations(
            project_name,
            context_filters=context_filters
        ))

My worry is not only the fact that the API doesn't provide that same support but the fact that not having that is now forcing the user to provide much more expensive implementations to get to the same result. In the new code you are now having to query ALL products and then filter them rather than doing it directly on the query

[BigRoy]

Just want to add the sidenote that in the original code in OpenPype going through the representations context field wasn't actually the way it was supposed to be done since it wouldn't be 100% reliable. E.g. it wouldn't survive a refactor to the AYON api because we can't rely on the 'legacy' context to have the keys we may expect now (e.g. folder instead of asset, etc.) because that context is persisted on publish instead of actually filtering towards the exact representation from its parent version, product, folder, etc.

As such, adding the ability to 'regex' filter would really apply to being able to regex query the folders first, then the products, etc.

Note that the regex ability came from MongoDB supporting that natively - I'm not sure there's any equivalent for the new backend?

For this particular case it'd mostly be an optimization now to indeed query the backend by some regex if we can, but going through folder -> product -> etc. should actually be considered a bugfix.