Add additional PromQL operators to synthetic load

prometheus / test-infra

Prometheus E2E benchmarking tool

Apache License 2.0

156 stars 67 forks source link

Add additional PromQL operators to synthetic load #747

Closed kushalShukla-web closed 1 week ago

kushalShukla-web commented 1 month ago

This PR enhances the synthetic load generation by incorporating additional PromQL operators in the 6_loadgen.yaml file:

Added binary arithmetic operators (joins) to cover more complex query types.
Included logical operators (and, or, unless) for better query testing coverage.
Added topk function to test query performance with ranked results.

kushalShukla-web commented 1 month ago

Solves #705

kushalShukla-web commented 1 month ago

Hi, I commented on a few, but they are all likely problemmatic. I recommend you change tack to use metrics exposed by the fake webserver, since there are a lot of them and prombench can control them.

Actually i have tested this on one of the Pull Request where the prombench was running.

bboreham commented 1 month ago

Ok but my recommendation remains the same.

bboreham commented 1 month ago

I have replaced some of the existing metrics with new ones from PromBench, such as:

How many of those did you count?

kushalShukla-web commented 4 weeks ago

Hi @bboreham all the queries having value above 2000 , and codelabz metric is having 52000 different variations .

bboreham commented 1 week ago

I trimmed down the newly-added queries a bit:

Slow down arithmetic_operation and logic_operator.
Took out a few queries to avoid overloading the server.
Stop querying _bucket series directly; those should be used by histogram_quantile or similar.
Use more realistic k parameters to topk.

I also trimmed down some pre-existing queries for balance, to retain about the same overall load on the server as before.