On one hand, the loadgen querier makes the benchmark result more apparent. For instance, the memory usage of the Collector when the querier is enabled is ~6.03GB but after disabling the querier, it drops down to ~3.38GB. Further, the number of samples exported in a single RPC batch drops down from ~156 to ~44. Also, the Go process's heap idle bytes drops down from 7.92G to ~2G.
Also, since GCP has cloud monitoring as the front-end that supports querying for prometheus metrics (alongside several other things), loadgen querier may seem as a burden in the benchmarking infrastructure.
For example:
Collector's Memory usage when querier is enabled:
Collector's Memory usage when querier is disabled:
On the other hand, loadgen querier may be required to get the idea about how the memory, CPU, and other metrics impact the performance of the collector when it is queried directly (for example, via any 3rd party monitoring and reporting instance [e.g. a custom grafana instance] set up to query the collector). In this case, having the querier component may seem to be relevant in prombench's infrastructure.
A potential solution to this is making the querier optional (via a flag). This way, if need be, the querier component can be enabled/disabled at will and this does not significantly impact the infrastructure setup.
This is a two-faced argument:
On one hand, the loadgen querier makes the benchmark result more apparent. For instance, the memory usage of the Collector when the querier is enabled is ~6.03GB but after disabling the querier, it drops down to ~3.38GB. Further, the number of samples exported in a single RPC batch drops down from ~156 to ~44. Also, the Go process's heap idle bytes drops down from 7.92G to ~2G.
Also, since GCP has cloud monitoring as the front-end that supports querying for prometheus metrics (alongside several other things), loadgen querier may seem as a burden in the benchmarking infrastructure.
For example:
Collector's Memory usage when querier is enabled:
Collector's Memory usage when querier is disabled:
On the other hand, loadgen querier may be required to get the idea about how the memory, CPU, and other metrics impact the performance of the collector when it is queried directly (for example, via any 3rd party monitoring and reporting instance [e.g. a custom grafana instance] set up to query the collector). In this case, having the querier component may seem to be relevant in prombench's infrastructure.
A potential solution to this is making the querier optional (via a flag). This way, if need be, the querier component can be enabled/disabled at will and this does not significantly impact the infrastructure setup.
Example: https://github.com/GoogleCloudPlatform/prometheus-test-infra/commit/008027a7aa287860bec08880e1a7f7901603611d (Disabling the loadgen querier via a flag)