albertisfu
commented
2 months ago I suspect Elastic will have a better solution. @albertisfu, do you know if Elastic has a way of pulling boosting scores from a file? Any ideas if it can be done from S3 or something like that?
I did some investigation on this:
The ability to store a script in a file to tweak relevance was removed since Elastic 6.0
This feature was replaced by stored scripts, as explained in the documentation: https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-scripting-using.html#script-stored-scripts
Documentation says:
You can store and retrieve scripts from the cluster state using the stored script APIs. Stored scripts reduce compilation time and make searches faster.
Using this feature, we would need to create a stored script via the API using a request like:
PUT _scripts/my-stored-script
{
"script":{
"lang":"painless",
"source":"return params.scores.containsKey(doc['docket_id'].value)",
"params":{
"scores":{
"cluster_id_1":1.5,
"cluster_id_2":0.8,
"cluster_id_3":2.0
}
}
}
}Then, in the query, we could use a script_score along with this stored script to enhance relevance:
{
"query": {
"script_score": {
"query": {
"match_all": {}
},
"script": {
"id": "my-stored-script",
}
}
}
}However, we should assess the performance of this approach, as I assume the stored script may need to handle millions of parameters (one score per document)? Another downside is that new documents won’t be included in the stored script’s parameters, so they won’t be boosted by the relevance script until the parameters are updated.
An alternative approach for implementing this feature in ES could be to use a rank_feature field and query.
This is the method ES recommends for applying static signals (similar to PageRank) to enhance relevance: https://www.elastic.co/guide/en/elasticsearch/reference/current/static-scoring-signals.html https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-rank-feature-query.html
This approach seems to be the best in terms of performance. According to the documentation:
Unlike the function_score query or other ways to change relevance scores, the rank_feature query efficiently skips non-competitive hits when the track_total_hits parameter is not true. This can dramatically improve query speed.
Since we currently have track_total_hits disabled in our queries, we would benefit from this performance improvement.
However, this approach would require adding a new rank_feature field to the documents. This field has the following constraints:
- It only supports single-valued fields with strictly positive values.
- It preserves 9 significant bits of precision, resulting in a relative error of about 0.4%.
The rank_feature field can then be used in queries like this:
{
"query": {
"bool": {
"must": {
"match_all": {}
},
"should": {
"rank_feature": {
"field": "pagerank",
"saturation": {
"pivot": 10
}
}
}
}
}
}The rank_feature score can only be used with the following mathematical functions:
- Saturation
- Logarithm
- Sigmoid
- Linear
One downside to this approach is that it requires adding a new field to the documents and populating it with values. If we only add this field to the parent documents, we could potentially update the new values relatively quickly using an asynchronous update_by_query request.
However, I’m unsure whether the ranking algorithm we'll use can compute scores for new documents at indexing time, or if it would require re-evaluating the entire corpus to compute fresh scores for all documents when new ones are added or removed. If the latter is the case, we’d need to perform a whole reindex for this field each time the scores are recomputed.
Let me know if you need more details about any of these approaches.
mlissner
mattdahl
Over in https://github.com/freelawproject/courtlistener/issues/4312#issuecomment-2339308370, we have a little discussion about a new network ranking algorithm that @mattdahl is working on.
He says:
I think this would be great. In the past, with Solr, we computed pagerank once/month, and it cranked out a CSV file that we used to boost results. It worked fine until we moved Solr to its own server and it became too difficult to move the CSV over to that server from the one that computed it.
I suspect Elastic will have a better solution. @albertisfu, do you know if Elastic has a way of pulling boosting scores from a file? Any ideas if it can be done from S3 or something like that?
Matt, if this is doable technically, I'd love to consider doing it. Can you share your paper? I've been wanting a better network rank for a while.
I think if we put this in place, we'd wind up with:
Any others I'm forgetting?
That's feeling like a lot, but maybe it's fine. Maybe we wouldn't need the time decay with this algo?