search

opensearch-project / opensearch-benchmark

OpenSearch Benchmark - a community driven, open source project to run performance tests for OpenSearch
https://opensearch.org/docs/latest/benchmark/
Apache License 2.0
111 stars 78 forks source link

Add calculate-recall parameter to vector search and skip calculating recall if number clients > cpu cores #626

Closed finnroblin closed 3 weeks ago

finnroblin commented 2 months ago

Description

When the number of search clients is greater than the number of cpu cores recall drops unexpectedly. @VijayanB is investigating the cause, but this PR throws a warning if num_clients > the cpu cores available and skips calculating the recall. It also adds a parameter calculate-recall to specify whether or not to calculate recall (either calculate-recall: true or calculate-recall: false).

Testing

Unit tests. Also tested on mac with 12 cores. Skips recall calculation if num_clients > 12, calculates recall otherwise. calculate-recall parameter works as expected. Will add unit tests in a second PR by EOD.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.

IanHoang commented 3 weeks ago

Had a sync with @VijayanB and @gkamat offline. We'll work with @VijayanB to better understand the issue and root cause it. Closing this as of no activity. Feel free to reopen if needed.