Setting a Filter with MemoryFilter.MinRelevance value, makes the Az Search request fail.

[luismanez]

I don't know if this is something we can do in Az search, but I'd like to get documents with Score/Relevance greater than a given value.

The SearchAsync (and AskAsync) has a Filter parameter, and there there is a MinRelevance property. However, this property is not used later in code, and that generates an Azure Search API request with the Filter parameter empty, so it fails. This snippet fails:

        var data = await _memory.SearchAsync(
            query: _question,
            index: Constants.AzureSearchIndexName,
            filter: new MemoryFilter { MinRelevance = 0.8f }
            limit: 10
        );

I'd like to have that Filter by Score, cos I'm having the following behaviour: I have indexed just 3 documents, 2 of them are related to Domain Driven Design, and the 3rd one is the wikipedia into about Spiderman. I ask then the following question to the AskAsync method: "When an Entity should be considered an aggregate in DDD"

In the relevant results, I'm getting Citations/Parts from the Spiderman document. Obviously, they are sorted below the ones related with DDD, but as when the Prompt is composed, facts are added depending on the Tokens size, you can end up sending facts from Spiderman document. If we can filter the Search query to say "only search documents having score > 0.8" will avoid sending Spiderman facts to the Prompt.

[anthonypuppo]

https://github.com/microsoft/kernel-memory/blob/b275662fdb6d5429f674d0ac8d9fb753054ef000/dotnet/CoreLib/MemoryStorage/AzureCognitiveSearch/AzureCognitiveSearchMemory.cs#L574-L581

As there's no conditions it builds a string of "()". An invalid OData filter, so the call to ACS fails. It should check whether there's any conditions in the filter and, only then, append to the conditions list.

I'll create a PR to fix this in a bit unless someone gets to it first.

[luismanez]

yeah, but is not just that. The MemoryFilter has a MinRelevance property that is not used at all, and not sure what was the idea behind that property, cos what I'd like to do is to filter my query, so I only get docs with a Relevance greater than a value...

[anthonypuppo]

I see what you mean now. Either ISemanticMemoryClient.SearchAsync should expose a minRelevance argument and pass it to SearchAsync or just take the value of MinRelevance from the filter and pass that to SearchAsync. I'd prefer the former as I'm not sure how you could reliably detect the intended min if multiple filters are passed (unless it defaults to the min/max).

[luismanez]

yeah, agreed. Do you know what to do with that value? asking cos as far as I know, Azure Search does not support to filter by Relevance/Score. Current code seems they are using that value to do some evaluation in memory using that value with a Cosine function 😮 ...

[anthonypuppo]

I'm pretty sure (as of right now) there's no support for querying and passing a minimum score value. Hopefully this gets added, would be a real neat feature, but unsure how it would work as (especially in hybrid scenarios) there's multiple scores that end up getting calculated. Possibly just a filter on the final, RRF calculated score?

Here's some more info on how the scoring works in case it's helpful for your use case (and why there's a score transformation step to convert back to cosine similarity): https://learn.microsoft.com/en-us/azure/search/hybrid-search-ranking

[luismanez]

Thanks for the info. Yeah, I'd be happy if I could just filter by the same Score value that now is returned in the current Az Search request (the search engine is already calculating a Score somehow, so I'd just want to do $filter=score gt 0.8... before the kernel-memory library groups the results by doc_id to return the Citations...)