Backend: add search and filter possibility on dataset endpoints

[jokiefer]

Environment

• Python version:
• MrMap version:

User Story

As an ApiClient, I want to search for dataset metadata records, so that i can find/filter them by given keywords, text search.

Acceptance criteria

• [x] Given I am an ApiClient, when I search by text 'krankenhaus', I want to get all dataset metadata records which are related with a keyword which contains this text or abstract contains or title contains.
• [x] Given I am an ApiClient, when I filter by keyword(s), I want to get only the dataset metadata records which are related to that keyword(s)

[jokiefer]

For now we can filter and search on the dataset metadata endpoint. The filter[search] param will follow the django default implementation. This is a simple icontains lookup on any defined field. No fulltext search without any ranking of search results.

@MrSnyder @Unraveler IF we need a real search enginge which returns the results with ranking information, i could implement a solution like this as a generic mixin for all viewsets. If you give me a 👍🏼 i would open a new issue and implement this feature.

[MrSnyder]

@jokiefer I think, this is a good idea. I believe that we will feel the need for ranking and full-text search in the (near) future, so it seems reasonable to evaluate the capabilities provided by Django+Postgres and provide an endpoint to access them.

We should keep in mind though, that adding full-text search can also result in degraded precision, as @armin11 pointed out.

:+1:

[jokiefer]

We should keep in mind though, that adding full-text search can also result in degraded precision, as @armin11 pointed out.

Yep - we keep this in mind - but i think this is configure able. On any viewset you should still have to configure the following params:

• search_fields (required)
• search_vector (optional) to allow configuring weight

So to control the precision, you could configure the search_vector as following:

class DatasetMetadataViewSet(ModelViewSet):
    ...
    search_fields = ('title', 'abstract', 'keywords__keyword')
    search_vector = SearchVector("keywords__keyword", weight="A") + SearchVector("title", weight="B") + SearchVector('abstract', weight='D')

The example above will only search in fields of the DatasetMetadata object. Related layers, featuretypes and services are also possible to search in deep.