ethpandaops / ethereum-metrics-exporter

A Prometheus exporter for Ethereum Execution & Consensus clients
100 stars 13 forks source link

Multiple series when EL version changes #107

Closed skylenet closed 3 months ago

skylenet commented 3 months ago

So the EL version is exposed via the following example series:

eth_exe_web3_client_version{ethereum_role="execution",module="web3",node_name="execution",version="erigon/3.00.0/linux-amd64/go1.22.6"} 1

When the version changes, it will add a new series, therefore having multiple series, which shouldn't be the case, otherwise we get bad version reports, like this:

image

At around ~14h (left side), the version did actually change. But as you can see, the old series with the old version kept being announced, creating this confusing graph that looks like that it was running 2 versions simultaneously. At around ~12h (right side of the graph), you see the drop happening because the metrics exporter got restarted.

How to reproduce

Possible fixes:

Drop any previously announced eth_exe_web3_client_version metrics whenever the version changes. Note that this problem/solution could also be relevant to other metrics.. E.g. the CL client version.