search

IntelLabs / fastRAG

Efficient Retrieval Augmentation and Generation Framework
Apache License 2.0
1.29k stars 116 forks source link

Load performance: latency/RPS #4

Closed Matthieu-Tinycoaching closed 1 year ago

Matthieu-Tinycoaching commented 1 year ago

Hi,

Have you load tested fastRAG pipeline with Colbert, PLAID and FiD on CPU instance?

Could you provide examples of latency, RPS relatively to CPU instance characteristics?

danielfleischer commented 1 year ago

We're currently working on these types of measurements. We'll publish some benchmarks in a blog, hopefully soon. Thanks for the question!

danielfleischer commented 1 year ago

I publish a blog, benchmarking DPR and ColBERT+PLAID on CPU; have a look, you may find it interesting.