search

quickwit-oss / quickwit

Cloud-native search engine for observability. An open-source alternative to Datadog, Elasticsearch, Loki, and Tempo.
https://quickwit.io
Other
6.99k stars 291 forks source link

Connect shard throughput #5151

Closed fulmicoton closed 2 days ago

fulmicoton commented 1 week ago

Tested on simian with a pathologic workload:

- chico(rate=40mb)
- chico(rate=10kb)*10

On the left, without any per-shard weighting. On the right, with per-shard weighting.

indexing throughput image

cpu usage image

github-actions[bot] commented 1 week ago

On SSD:

Average search latency is 1.0x that of the reference (lower is better).
Ref run id: 2268, ref commit: 9fddb686f0024d58fc18dc77c44dc5358bbf0518
Link

On GCS:

Average search latency is 1.05x that of the reference (lower is better).
Ref run id: 2269, ref commit: 9fddb686f0024d58fc18dc77c44dc5358bbf0518
Link

fulmicoton commented 2 days ago

Closed and reopened as #5196