Metric `validator_monitor_beacon_block_total` starts from 1

[ghost]

Description

I want to create a table with blocks, proposed by our validators, but I found out that validator_monitor_beacon_block_total starts immediately from 1. This is a problem because it's really hard to detect exactly this first appearance in Prometheus via changes, rate and other functions.

Metric validator_monitor_beacon_block_total starts from 1 and before that event it doesn't exist.

Currently, I make my query, which consist of two parts:

• Find first appearance of metric started from 1. Use query like validator_monitor_beacon_block_total{...} unless validator_monitor_beacon_block_total{...} offset 1m) ==1
• Find all metric changes. Use query like changes(validator_monitor_beacon_block_total{...}[5m]) > 0

Full query looks like this:

1000* min by (validator) (timestamp( (validator_monitor_beacon_block_total{...} unless validator_monitor_beacon_block_total{...} offset 1m) ==1 or changes(validator_monitor_beacon_block_total{...}[5m]) > 0)) 

And this first part for detecting exactly first metric appearance is really annoying and sometimes unstable. It could be a more way easy if this metric just started from 0 and appeared at lighthouse startup: I could use only changes() query.

The core problem: all metrics related to block proposer changes really rare, and every change is significant, but in current approach it's hard to detect first appearance of this metrics in Prometheus.

Version

lighthouse --version
Lighthouse v3.0.0-18c61a5
BLS library: blst-portable
SHA256 hardware acceleration: false
Specs: mainnet (true), minimal (false), gnosis (true)

Present Behaviour

Metric validator_monitor_beacon_block_total only appears after first block proposal and starts from 1

Expected Behaviour

How should the application behave?

Metric validator_monitor_beacon_block_total should start from 0 at startup time

[tthebst]

I think the issue is that metric variables in lighthouse are defined as lazy static variables. This means validator_monitor_beacon_block_total is therefore created on first access where it is immediatly incremented and you won't see any 0 -> 1 transition in Prometheus.

A possible solution for this is to call lazy_static::initialize(& VALIDATOR_MONITOR_BEACON_BLOCK_TOTAL); when the beacon node starts.