elastic / elasticsearch

Free and Open Source, Distributed, RESTful Search Engine
https://www.elastic.co/products/elasticsearch
Other
1.21k stars 24.84k forks source link

New Rally `low-latency` challenge in `tsdb` track #108091

Open salvatore-campagna opened 6 months ago

salvatore-campagna commented 6 months ago

Description

We need to evaluate the impact of ZSTD compression on low latency queries such as get and mget and for this reason we need a new challenge running in Rally nightlies.

After evaluating Rally benchmark results we identified a potential regression affecting low latency queries using ZSTD to compress stored fields. The major field contributing to increased fetch latency is the _soruce field which exists if synthetic source is not enabled. This is the case for the tsdb track when executing with index.mode = standard. Moreover we need to use the tsdb track because document ids are not random and running queries such as get and mget results in the ability to re-run tests over time with document ids known upfront.

Moreover we need to use a new challenge because:

By having a separate challenge we make sure only the new low-latency performance test fails without affecting other benchmarks whose results are still valid.

elasticsearchmachine commented 6 months ago

Pinging @elastic/es-storage-engine (Team:StorageEngine)

martijnvg commented 6 months ago

I think 8.13-best-compression vs 8.14-best-compression would also be a good comparison.

salvatore-campagna commented 6 months ago

I need to re-run the tests above since they have errors. search_by_doc_id runs fine while get and mget fail, which is a bit surprising. I need to verify that assertions are enabled, because I would expect search_by_doc_id to fail too.

salvatore-campagna commented 5 months ago

Attaching test results params-8-13-tsdb-low-latency-best-compression.json params-8-13-tsdb-low-latency-best-speed.json params-8-14-tsdb-low-latency-best-compression.json params-8-14-tsdb-low-latency-best-speed.json

tsdb-low-latency-best_compression-8-13-vs-best_compression-8-14.txt tsdb-low-latency-best_speed-8-13-vs-best_speed-8-14.txt

salvatore-campagna commented 5 months ago

Comparing the best compression option using Elasticsearch 8.13 and Elasticsearch 8.14 we can see results that seems a bit contradicting, with some GET queries being faster in 8.14 and other being slower (at higher latency percentiles). I think this is due to the fact that queries are very low latency queries taking less than 10 ms. This makes the test sensible to the environment and to test noise.

In the best speed scenario, intead, results seems to suggest that ZSTD is faster.