Design and develop a new tool to evaluate the quality of a set of search results

[jjacob7734]

💡 Description

Design and develop a software tool that can assign a score to a set of search results to quantify the quality of the result set. The inputs to the tool are a set of search results and an a labeling of the top 10 documents in the search result set on a scale of 0 to 5 where 0 means completely irrelevant and 5 means extremely highly relevant. The expectation is that the documents of high relevance will appear at or near the top of the result set for a successful search. Only the top 10 documents need to be considered for evaluation in this iteration.

[tloubrieu-jpl]

@jjacob7734 is making progress on this ticket with a single test test suite (Cassini search).

[jjacob7734]

The best metric I've found to evaluate the quality of a search technology is the Normalized Discounted Cumulative Gain (NDCG). That metric and other common ones are succinctly described here: https://ml-compiled.readthedocs.io/en/latest/metrics.html.

[jjacob7734]

I am using this implementation in the scikit-learn module: https://scikit-learn.org/stable/modules/generated/sklearn.metrics.ndcg_score.html

[jordanpadams]

Not completed prior to project pause. Moving to icebox