Nearest neighbor operator fails in custom searcher

[danitico]

Describe the bug We are trying to do a nearest neighbor search inside our custom searcher. We have followed indications in https://github.com/vespa-engine/vespa/issues/23875 and we confirm that we have the following:

• A ranking profile where the query tensor is defined
• That we are using the correct ranking profile
• That we are setting programatically the embedding as a ranking feature property

After doing of all these, we see the following error:

Error in search reply.: NearestNeighborTerm(document_embedding_es, q): Query tensor was not found in request context. Returning empty blueprint

However doing this type of search running the query against the /search endpoint, we are able to collect the expected documents. As you can see in the following files, we are setting the same parameters for both queries but the one in the custom searcher fails.

• Trace file when using custom searcher
• Trace file when using /search endpoint

To Reproduce Steps to reproduce the behavior:

1. Set up a nearest neighbor operator inside a custom searcher
2. Do a query and receive 0 results (see error)
3. Compare the behaviour with the default /search endpoint

Expected behavior It should work correctly in both scenarios

Screenshots No applicable

Environment (please complete the following information):

• OS: macOS
• Infrastructure: self-hosted
• Versions 13.5

Vespa version 8.202.11

Additional context https://github.com/vespa-engine/vespa/issues/23875

[jobergum]

We have multiple working examples of using the nearestNeighbor query item in a searcher, so we would appreciate the exact details on how to reproduce. That should consist of:

• A minimalistic schema(s)
• A searcher implementation that reproduces

Here are a few sample apps that use the nearestNeighbor query item:

• https://github.com/vespa-engine/sample-apps/blob/master/msmarco-ranking/src/main/java/ai/vespa/examples/searcher/RetrievalModelSearcher.java
• https://github.com/vespa-cloud/vespa-documentation-search/blob/main/src/main/java/ai/vespa/cloud/docsearch/SuggestionSearcher.java
• https://github.com/vespa-cloud/cord-19-search/blob/main/src/main/java/ai/vespa/example/cord19/searcher/RelatedArticlesByNNSearcher.java

[danitico]

@jobergum I need to retire my words. The problem was the following:

• We have a tensor of type bfloat16 of 768 dimensions
• The one declared at the ranking profile is tensor of type float of 768 dimensions.

So the behaviour for tensor type checking is different in both scenarios (custom searcher vs /search endpoint). I suppose that inside the searchers defined in the /search endpoint there must be a casting to that type

[jobergum]

When you use the search endpoint and pass a string tensor, that is converted to the tensor type specified in the schema or the query profile types. When using a searcher, and adding a different tensor type, this is the result.