Prometheus / Monitoring metrics

[denisgranha]

The mainnet chain is growing everyday and problems that were small in the past are now more important with bigger chain size, bandwith, etc.

Geth introduced in 1.9.0 version Prometheus metrics https://blog.ethereum.org/2019/07/10/geth-v1-9-0/

In order to have better alerting, monitoring and a better understanding of network health we need better metrics

[eduadiez]

Totally agree!

[adria0]

@dvdplm @ordian @sorpaas based on your experience, which internal "states"/variables (of any subsystem) could be more interesting to have telemetry activated?

[ordian]

Reposting from an internal discussion that took place a year ago:

• Database metrics
  • Disk usage over blocks/time
  • Comparable metrics for fast-pruned nodes, archive nodes, fatdb nodes, tracing nodes
  • Breakdown disk usage in block headers, block bodies, receipts/logs, indices, etc.
• Memory metrics
  • What’s memory used for (breakdown?)
  • How does memory/caching optimize node performance, how much can we reduce IO here
  • What are optimal settings for an Ethereum node
• CPU metrics
  • Running a network entirely in memory to eliminate IO
  • How much transactions per second can a node handle (processing, executing transactions & sealing blocks)
  • blocks per second, transactions per second, gas per second
  • How can we optimize settings for maximum throughput
• Other metrics
  • JSON-RPC APIS (HTTP, WS)
  • Transaction queue?
  • Mining latency, getwork, stratum?
  • Networking metrics, how many peers can we handle, bandwidth requirements

[adria0]

WIP in https://github.com/openethereum/openethereum/tree/adria0/prometheus