search

elastic / elasticsearch

Free and Open, Distributed, RESTful Search Engine
https://www.elastic.co/products/elasticsearch
Other
69.41k stars 24.56k forks source link

Make matched_queries available in the Painless script context #85629

Open scurtis-disco opened 2 years ago

scurtis-disco commented 2 years ago

Description

As noted in the forum here , I was attempting to make use of my named queries in a Painless script, in order to determine definitively which clause(s) matched a given query. I was surprised to find that none of my contextual options, like doc['matched_queries'], and params['_source']['matched_queries'], came back with anything, since I could see them in the output using Kibana.

I found a Stack Overflow question that led to another forum discussion from 2017, that stated that matched_queries are apparently added post-fetch, and presumably after all scripts and scoring are run.

Our current hacky way of identifying the matched clauses, assigning each clause a unique value in the source field and then inspecting the resulting composite score when the script executes, is fraught with peril and very fragile, given that each new clause added (we're up to 8 ATM) means adjusting our script code to factor in the new value every time. My proposal is that, by whatever means possible, these be "promoted" to a higher level in the execution chain so that they can be available in the scoring/scripting phase.

elasticmachine commented 2 years ago

Pinging @elastic/es-core-infra (Team:Core/Infra)

nemphys commented 1 year ago

This could indeed by very helpful in some use cases, +1.

elasticsearchmachine commented 1 year ago

Pinging @elastic/es-search (Team:Search)

rjernst commented 1 year ago

I've relabeled this for the Search team to consider. This issue has nothing to do with Painless the language, nor scripting infrastructure. It is a decision the Search team will have to make as to the feasibility of providing matched queries within ScoreScripts.

scurtis-disco commented 1 year ago

I've relabeled this for the Search team to consider. This issue has nothing to do with Painless the language, nor scripting infrastructure. It is a decision the Search team will have to make as to the feasibility of providing matched queries within ScoreScripts.

Thanks very much, @rjernst .

benwtrent commented 1 month ago

Matched queries are all calculated way after the query phrase. The information is calculated during fetch.

Attempting to pass this information along during the query phase would be fairly expensive and require a good amount of refactoring.

elasticsearchmachine commented 1 month ago

Pinging @elastic/es-search-relevance (Team:Search Relevance)

scurtis-disco commented 1 month ago

Matched queries are all calculated way after the query phrase. The information is calculated during fetch.

Attempting to pass this information along during the query phase would be fairly expensive and require a good amount of refactoring.

Pardon my ignorance (and the fact that I am returning to this more than 2 years after the original issue was opened), but the Painless script context doesn't only exist during the query phase, does it? I mean, on this project I referenced, we were calling custom scoring functions which implies results in hand, i.e., after the query phase has run.

Full disclosure: I am not on that project any more (or even with the company I was with at that time), so I have no way of going back and validating what we were doing at the time.