celestiaorg / celestia-app

PoS application for the consensus portion of the Celestia network. Built using celestia-core (fork of CometBFT) and the cosmos-sdk
https://celestia.org
Apache License 2.0
328 stars 261 forks source link

After upgrade to v1.11.0, the consensus_validator_missed_blocks metric is no longer available. #3619

Open GImbrailo opened 4 days ago

GImbrailo commented 4 days ago

Summary of Bug

After upgrading our mochanet validator node to celestia-appd v1.11.0, we no longer see the metric consensus_validator_missed_blocks. We use this metric in our monitoring and alerting systems to alert if/when our validator starts missing blocks.

I compared v1.9.0 and v1.11.0 of celestia-appd, as well as v1.35.0-tm-v0.34.29 and v1.36.0-tm-v0.34.29 of celestia-core, and I have not seen any obvious changes to metrics.

Version

v1.11.0

Steps to Reproduce

If your node exposes metrics on the default port, then you can run the following curl -s localhost:26660/metrics | grep validator_missed_blocks

On v1.9.0, you will see something like the following

# HELP celestia_consensus_validator_missed_blocks Total missed blocks for a validator
# TYPE celestia_consensus_validator_missed_blocks gauge
celestia_consensus_validator_missed_blocks{chain_id="celestia",validator_address="<address>",version="1.9.0"} 4

On v1.11.0, there will be no lines that contain consensus_validator_missed_blocks