search

harsha-simhadri / big-ann-benchmarks

Framework for evaluating ANNS algorithms on billion scale datasets.
https://big-ann-benchmarks.com
MIT License
336 stars 112 forks source link

the definition of QPS in internet services is different #24

Open nk2014yj opened 3 years ago

nk2014yj commented 3 years ago

In most internet services, QPS means the server can process how much requests in parallel (every second). That is not only affected by the cost time of each request. In faiss-lib is using like "pragma omp parallel " methods to reduce runing time, which does not promote the ability of process requests in each second. The results of our experiment showed that use parallel instructions may reduce the QPS when the CPUs are fully loaded. would you considering using the commonly definition of QPS, and provideing a rpc-service to compare the true ability of processing search-requests.

sourcesync commented 3 years ago

Are you referring to our existing formula for measuring throughput?

(latency to search batch of queries) / ( number of queries in a batch )

On Mon, Aug 23, 2021 at 5:06 AM nk2014yj @.***> wrote:

In most internet services, QPS means the server can process requests in parallel (every second). That is not only affected by the cost time of each request. In faiss-lib is using like "pragma omp parallel " methods to reduce runing time, which does not promote the ability of process requests in each second. The results of our experiment showed that use parallel instructions may reduce the QPS. would you considering using the commonly definition of QPS, and provideing a rpc-service to compare the true ability to process search-requests of each algorithm.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/harsha-simhadri/big-ann-benchmarks/issues/24#issuecomment-903704562, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADL6CMKXBHZJK2NAJB4OV3T6I2VXANCNFSM5CUNFRIQ .