search

trinodb / benchto

Framework for running macro benchmarks in a clustered environment
Apache License 2.0
28 stars 30 forks source link

Normalize metrics from measurements #45

Open nineinchnick opened 2 years ago

nineinchnick commented 2 years ago

This PR extracts name and unit from measurements into another metrics table. Such metrics also have attributes, which allow the removal of prefixes and suffixes from metric names and make it easier to filter some metrics when building reports.

The denormalized measurement.name and measurement.unit columns are not removed, but the name format is changed from cluster-cpu_max to cpu {scope=cluster,aggregate=max}.

This change matches how other metric monitoring system stores data, using labels (Prometheus) or attributes (OpenTelemetry).

nineinchnick commented 1 year ago

@przemekak @sopel39 @radek-starburst PTAL. There are breaking changes here but I tried to preserve some backward compatibility.

sopel39 commented 1 year ago

@nineinchnick are you working on this? Is this required for benchmarking infra

nineinchnick commented 1 year ago

@nineinchnick are you working on this? Is this required for benchmarking infra

I'm not, since it's not required. It would only make report queries a bit simpler, as it would have to parse metric names to split out prefixes.

Because this is a breaking change, I want to prepare responses to all the comments I got, but it's a low priority.