search

opensearch-project / opensearch-benchmark-workloads

Official workloads used by OpenSearch Benchmark (OSB)
https://opensearch.org/docs/latest/benchmark/
11 stars 58 forks source link

[FEATURE] Add recall calculation as part of benchmark summary for vectorsearch workloads. #282

Open akashsha1 opened 2 months ago

akashsha1 commented 2 months ago

Is your feature request related to a problem?

Vector search workloads presently do not support recall calculation in the summary section.

What solution would you like?

VectorSearch outcomes rely on recall, which is an important metric in determining if the search was successful. It would be ideal to include this metric, as new workloads and benchmarks in this space evolve (e.g. semantic search)

What alternatives have you considered?

The current alternative is to manually calculate the recall offline based on the data being published, or using opensearch queries to aggregrate the information.

gkamat commented 2 months ago

Will look into this issue further shortly, whether this could be just another metric that is reported.