Open razilevin opened 1 year ago
Hacked like this to make work. Here is the definition of the feature. Any feedback?
{
"name": "vector_simularity",
"params": [
"embedding"
],
"template_language": "mustache",
"template": {
"function_score": {
"script_score": {
"script": {
"source": """
List parseArrayOfFloats(def aryOfFloats) {
def x = aryOfFloats.substring(1, aryOfFloats.length() - 1);
def z = new StringTokenizer(x, ",");
def y = new ArrayList();
while(z.hasMoreTokens()) {
y.add(Float.parseFloat((String)z.nextToken()));
}
return y;
}
return cosineSimilarity(parseArrayOfFloats(params.query_vector), 'base_name_vector') + 1.0;
""",
"params": {
"query_vector": "{{#toJson}}embedding{{/toJson}}"
}
}
}
}
}
}
I believe that the problem is that the original query is not structured correctly. The template
can be a deeply nested query, or it can be a string. In order to have the toJson
work correctly, it needs to be a string:
{
"name": "vector_simularity",
"params": [
"embedding"
],
"template_language": "mustache",
"template": "{\"function_score\": {\"script_score\": {\"script\": {\"source\": \"1 + cosineSimilarity(params.query_vector, doc['base_name_vector'])\", \"params\": {\"query_vector\": {{#toJson}}embedding{{/toJson}}}}}}"
}
}
Note that all the "
are escaped and the {{#toJson}}embedding{{/toJson}}
is not enclosed in quotes
Trying to use embeddings to compute cosine similarity. The problem I am getting is there no way to pass the embedding as a param to invoke the following feature during logging.
I got the idea to use
toJson
mustache template from another post which seems to match what I am tying to do https://github.com/o19s/elasticsearch-learning-to-rank/issues/338I get the following error when running the query
Please note a query like the following works as expected