newHeads eth_subscribe has 2-15 seconds delay although Geth only 2-3 seconds

fridary commented 4 months ago

Erigon version: erigon version 2.59.3-088fd8ef OS & Version: Ubuntu 20.04.6 LTS, CPU E5-2695 v4, 36 cores, 192 Gb RAM Disk: NVMe HPE MT006400KWHAE 6.4 Tb Erigon Command (with flags/config): ExecStart=/home/root/erigon/build/bin/erigon --internalcl --datadir=/disk_sde/erigon --ws --http.api=eth,erigon,engine,web3,net,debug,trace,txpool --authrpc.jwtsecret=/home/root/erigon/jwtsecret --prune.h.before=13916166 --prune.r.before=13916166 --prune.t.before=13916166 --prune.c.before=13916166 --torrent.download.rate=128mb --rpc.batch.limit=5000 Chain/Network: mainnet

I have a server that is running 2 nodes: Erigon and Geth (snap). Both are running on the same disk disk_sde, everything is the same. But Geth receives newHeads (new blocks) by eth_subscribe using websockets much faster. Any ideas what's wrong?

Last journal logs:

-- Reboot --
may 31 01:08:00 root-pc systemd[1]: Started Erigon Node.
may 31 01:08:02 root-pc erigon[4745]: [INFO] [05-31|01:08:02.493] logging to file system                   log dir=/disk_sde/erigon/logs file prefix=erigon log level=info json=false
may 31 01:08:02 root-pc erigon[4745]: [INFO] [05-31|01:08:02.494] Build info                               git_branch=HEAD git_tag=v2.59.3 git_commit=088fd8ef69389a72da6faa0fc7903a4ba5726911
may 31 01:08:02 root-pc erigon[4745]: [INFO] [05-31|01:08:02.495] Starting Erigon on Ethereum mainnet...
may 31 01:08:02 root-pc erigon[4745]: [INFO] [05-31|01:08:02.500] Maximum peer count                       ETH=100 total=100
may 31 01:08:02 root-pc erigon[4745]: [INFO] [05-31|01:08:02.506] starting HTTP APIs                       port=8545 APIs=eth,erigon,engine,web3,net,debug,trace,txpool
may 31 01:08:02 root-pc erigon[4745]: [INFO] [05-31|01:08:02.506] torrent verbosity                        level=WRN
may 31 01:08:04 root-pc erigon[4745]: [INFO] [05-31|01:08:04.615] Set global gas cap                       cap=50000000
may 31 01:08:04 root-pc erigon[4745]: [INFO] [05-31|01:08:04.722] [Downloader] Running with                ipv6-enabled=true ipv4-enabled=true download.rate=128mb upload.rate=4mb
may 31 01:08:04 root-pc erigon[4745]: [INFO] [05-31|01:08:04.742] Opening Database                         label=chaindata path=/disk_sde/erigon/chaindata
may 31 01:08:04 root-pc erigon[4745]: [INFO] [05-31|01:08:04.747] [db] open                                lable=chaindata sizeLimit=12TB pageSize=8192
may 31 01:08:04 root-pc erigon[4745]: [INFO] [05-31|01:08:04.757] Initialised chain configuration          config="{ChainID: 1, Homestead: 1150000, DAO: 1920000, Tangerine Whistle: 2463000, Spurious Dragon: 2675000, Byzantium: 4370000, Constantinople: 7280000, Petersburg: 7280000, Istanbul: 9069000, Muir Glacier: 9200000, Berlin: 12244000, London: 12965000, Arrow Glacier: 13773000, Gray Glacier: 15050000, Terminal Total Difficulty: 58750000000000000000000, Merge Netsplit: <nil>, Shanghai: 1681338455, Cancun: 1710338135, Prague: <nil>, Osaka: <nil>, Engine: ethash, NoPruneContracts: map[0x00000000219ab540356cBB839Cbe05303d7705Fa:true]}" genesis=0xd4e56740f876aef8c010b86a40d5f56745a118d0906a34e69aec8c0db1cb8fa3
may 31 01:08:54 root-pc erigon[4745]: [INFO] [05-31|01:08:54.853] Initialising Ethereum protocol           network=1
may 31 01:08:54 root-pc erigon[4745]: [INFO] [05-31|01:08:54.853] Disk storage enabled for ethash DAGs     dir=/disk_sde/erigon/ethash-dags count=2
may 31 01:08:56 root-pc erigon[4745]: [INFO] [05-31|01:08:56.304] Starting private RPC server              on=127.0.0.1:9090
may 31 01:08:56 root-pc erigon[4745]: [INFO] [05-31|01:08:56.304] new subscription to logs established
may 31 01:08:56 root-pc erigon[4745]: [INFO] [05-31|01:08:56.304] [Checkpoint Sync] Requesting beacon state uri=https://mainnet-checkpoint-sync.attestant.io/eth/v2/debug/beacon/states/finalized
may 31 01:08:56 root-pc erigon[4745]: [INFO] [05-31|01:08:56.308] [txpool] Started
may 31 01:09:00 root-pc erigon[4745]: [INFO] [05-31|01:09:00.530] rpc filters: subscribing to Erigon events
may 31 01:09:00 root-pc erigon[4745]: [INFO] [05-31|01:09:00.530] new subscription to newHeaders established
may 31 01:09:00 root-pc erigon[4745]: [INFO] [05-31|01:09:00.530] New txs subscriber joined
may 31 01:09:00 root-pc erigon[4745]: [INFO] [05-31|01:09:00.531] Reading JWT secret                       path=/home/root/erigon/jwtsecret
may 31 01:09:00 root-pc erigon[4745]: [INFO] [05-31|01:09:00.532] JsonRpc endpoint opened                  ws=true ws.compression=true grpc=false websocket.url=127.0.0.1:8546 http.url=127.0.0.1:8545
may 31 01:09:00 root-pc erigon[4745]: [WARN] [05-31|01:09:00.535] NAT ExternalIP resolution has failed, try to pass a different --nat option err="no UPnP or NAT-PMP router discovered"
may 31 01:09:00 root-pc erigon[4745]: [INFO] [05-31|01:09:00.536] HTTP endpoint opened for Engine API      url=127.0.0.1:8551 ws=true ws.compression=true
may 31 01:09:00 root-pc erigon[4745]: [WARN] [05-31|01:09:00.536] NAT ExternalIP resolution has failed, try to pass a different --nat option err="no UPnP or NAT-PMP router discovered"
may 31 01:09:00 root-pc erigon[4745]: [INFO] [05-31|01:09:00.537] Started P2P networking                   version=67 self=enode://2734723951d34a6e40dd4c94c93fa97e8c5fe58fca385c9df3cce54323a8d0c3fcffa1dcef40da5f877255c03153605c3b97de8b00ed761fbf81e452155f9fbb@127.0.0.1:30304 name=erigon/v2.59.3-088fd8ef/linux-amd64/go1.22.1
may 31 01:09:00 root-pc erigon[4745]: [INFO] [05-31|01:09:00.537] Started P2P networking                   version=68 self=enode://2734723951d34a6e40dd4c94c93fa97e8c5fe58fca385c9df3cce54323a8d0c3fcffa1dcef40da5f877255c03153605c3b97de8b00ed761fbf81e452155f9fbb@127.0.0.1:30303 name=erigon/v2.59.3-088fd8ef/linux-amd64/go1.22.1
may 31 01:09:00 root-pc erigon[4745]: [INFO] [05-31|01:09:00.553] [1/12 Snapshots] Requesting downloads
may 31 01:09:04 root-pc erigon[4745]: [INFO] [05-31|01:09:04.055] [snapshots:download] Blocks Stat         blocks=19895k indices=19895k alloc=3.7GB sys=5.5GB
may 31 01:09:32 root-pc erigon[4745]: [INFO] [05-31|01:09:32.849] Static peers                             len=0
may 31 01:09:32 root-pc erigon[4745]: [INFO] [05-31|01:09:32.851] [Sentinel] Sentinel started              app=caplin enr=enr:-Ly4QIDi0f48s885fcdE4klJSbCBvjKZbO-Y11i5A5SYprzCZGTtbwy03PpUXJS-nufJXeH6ddAU3ETMFypX2lZsdPABh2F0dG5ldHOIAAAAAAAAAACEZXRoMpBqlaGpBAAAAP__________gmlkgnY0gmlwhH8AAAGJc2VjcDI1NmsxoQNu67INfIIUNcgV8idbt_k5EEfiBIwS3mSYqe7BrHu1l4hzeW5jbmV0cwCDdGNwgg-hg3VkcIIPoA
may 31 01:09:32 root-pc erigon[4745]: [INFO] [05-31|01:09:32.851] Started Ethereum 2.0 Gossip Service      app=caplin
may 31 01:09:38 root-pc erigon[4745]: [INFO] [05-31|01:09:38.199] [Caplin] starting clstages loop          app=caplin
may 31 01:09:38 root-pc erigon[4745]: [INFO] [05-31|01:09:38.200] Starting downloading History             app=caplin stage=DownloadHistoricalBlocks from=9190176
may 31 01:09:39 root-pc erigon[4745]: [INFO] [05-31|01:09:39.340] Backfilling finished
may 31 01:09:43 root-pc erigon[4745]: [INFO] [05-31|01:09:43.202] Ready to insert history, waiting for sync cycle to finish app=caplin stage=DownloadHistoricalBlocks
may 31 01:09:43 root-pc erigon[4745]: [INFO] [05-31|01:09:43.204] [Caplin] Forward Sync                    app=caplin stage=ForwardSync from=9190176 to=9190246
may 31 01:10:14 root-pc erigon[4745]: [INFO] [05-31|01:10:14.273] [Caplin] Forward Sync                    app=caplin stage=ForwardSync progress=9190218 distance-from-chain-tip=5m36s estimated-time-remaining=20s
may 31 01:10:32 root-pc erigon[4745]: [INFO] [05-31|01:10:32.859] P2P                                      app=caplin peers=37
may 31 01:10:48 root-pc erigon[4745]: [INFO] [05-31|01:10:48.360] [Caplin] Forward Sync                    app=caplin stage=ForwardSync progress=9190251 distance-from-chain-tip=-1m0s estimated-time-remaining=0s
may 31 01:10:59 root-pc erigon[4745]: [INFO] [05-31|01:10:59.154] [3/12 Senders] Started                   from=19985477 to=19985546
may 31 01:10:59 root-pc erigon[4745]: [INFO] [05-31|01:10:59.312] [4/12 Execution] Blocks execution        from=19985477 to=19985546
may 31 01:11:16 root-pc erigon[4745]: [INFO] [05-31|01:11:16.818] [4/12 Execution] Completed on            block=19985546
may 31 01:11:16 root-pc erigon[4745]: [INFO] [05-31|01:11:16.818] [5/12 HashState] Promoting plain state   from=19985477 to=19985546
may 31 01:11:16 root-pc erigon[4745]: [INFO] [05-31|01:11:16.818] [5/12 HashState] Incremental promotion   from=19985477 to=19985546 codes=true csbucket=AccountChangeSet
may 31 01:11:18 root-pc erigon[4745]: [INFO] [05-31|01:11:18.338] [5/12 HashState] Incremental promotion   from=19985477 to=19985546 codes=false csbucket=AccountChangeSet
may 31 01:11:21 root-pc erigon[4745]: [INFO] [05-31|01:11:21.935] [5/12 HashState] Incremental promotion   from=19985477 to=19985546 codes=false csbucket=StorageChangeSet
may 31 01:11:26 root-pc erigon[4745]: [INFO] [05-31|01:11:26.498] [6/12 IntermediateHashes] Generating intermediate hashes from=19985477 to=19985546
may 31 01:11:32 root-pc erigon[4745]: [INFO] [05-31|01:11:32.851] P2P                                      app=caplin peers=56
may 31 01:11:45 root-pc erigon[4745]: [INFO] [05-31|01:11:45.088] [7/12 CallTraces] Pruned call trace intermediate table from=19895477 to=19895545
may 31 01:11:54 root-pc erigon[4745]: [INFO] [05-31|01:11:54.849] [p2p] GoodPeers                          eth68=1
may 31 01:11:55 root-pc erigon[4745]: [INFO] [05-31|01:11:55.289] [mem] memory stats                       Rss=13.6MB Size=0B Pss=13.6MB SharedClean=2.9KB SharedDirty=0B PrivateClean=3.6MB PrivateDirty=10.0MB Referenced=13.6MB Anonymous=9.7MB Swap=0B alloc=10.0GB sys=10.5GB
may 31 01:11:56 root-pc erigon[4745]: [INFO] [05-31|01:11:56.305] [txpool] stat                            pending=0 baseFee=0 queued=1481 alloc=10.0GB sys=10.5GB
may 31 01:12:02 root-pc erigon[4745]: [INFO] [05-31|01:12:02.820] RPC Daemon notified of new headers       from=19985477 to=19985546 amount=69 hash=0x0190483e6525caf247f627a555a5243d0dff4486d9abd7cf88f5f5a232ce903d header sending=105.563µs log sending=269ns
may 31 01:12:02 root-pc erigon[4745]: [INFO] [05-31|01:12:02.821] head updated                             hash=0x0190483e6525caf247f627a555a5243d0dff4486d9abd7cf88f5f5a232ce903d number=19985546
may 31 01:12:32 root-pc erigon[4745]: [INFO] [05-31|01:12:32.852] P2P                                      app=caplin peers=65
may 31 01:13:32 root-pc erigon[4745]: [INFO] [05-31|01:13:32.852] P2P                                      app=caplin peers=67
may 31 01:13:46 root-pc erigon[4745]: [INFO] [05-31|01:13:46.735] [4/12 Execution] Completed on            block=19985560
may 31 01:13:51 root-pc erigon[4745]: [INFO] [05-31|01:13:51.225] RPC Daemon notified of new headers       from=19985546 to=19985560 amount=14 hash=0xfedb56f87b3b3f6978cd185ef140e54ac3eae7898ad1bd74482ff4b7427fbe9e header sending=55.332µs log sending=379ns
may 31 01:13:51 root-pc erigon[4745]: [INFO] [05-31|01:13:51.225] head updated                             hash=0xfedb56f87b3b3f6978cd185ef140e54ac3eae7898ad1bd74482ff4b7427fbe9e number=19985560
may 31 01:14:21 root-pc erigon[4745]: [INFO] [05-31|01:14:21.767] [4/12 Execution] Completed on            block=19985563
may 31 01:14:23 root-pc erigon[4745]: [INFO] [05-31|01:14:23.005] RPC Daemon notified of new headers       from=19985560 to=19985563 amount=3 hash=0x38dcadd6c0dbb3a4021a9f4b89a85a862d8746e054648105c63d7db344ab0e06 header sending=19.613µs log sending=327ns
may 31 01:14:23 root-pc erigon[4745]: [INFO] [05-31|01:14:23.005] head updated                             hash=0x38dcadd6c0dbb3a4021a9f4b89a85a862d8746e054648105c63d7db344ab0e06 number=19985563
may 31 01:14:31 root-pc erigon[4745]: [INFO] [05-31|01:14:31.094] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
may 31 01:14:31 root-pc erigon[4745]: [INFO] [05-31|01:14:31.782] RPC Daemon notified of new headers       from=19985563 to=19985564 amount=1 hash=0x7d386a065d75f54a0674ae4c404f40ba60a9532fcd0adbfa3631f54a71235aa6 header sending=17.422µs log sending=396ns
may 31 01:14:31 root-pc erigon[4745]: [INFO] [05-31|01:14:31.783] head updated                             hash=0x7d386a065d75f54a0674ae4c404f40ba60a9532fcd0adbfa3631f54a71235aa6 number=19985564
may 31 01:14:32 root-pc erigon[4745]: [INFO] [05-31|01:14:32.851] P2P                                      app=caplin peers=63
may 31 01:14:42 root-pc erigon[4745]: [INFO] [05-31|01:14:42.751] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
may 31 01:14:43 root-pc erigon[4745]: [INFO] [05-31|01:14:43.177] RPC Daemon notified of new headers       from=19985564 to=19985565 amount=1 hash=0xa2a051d3a8e0ae5edeedd59e3fa5c350c8cd676cf773dbb09d45a74d3d72e696 header sending=16.491µs log sending=408ns
may 31 01:14:43 root-pc erigon[4745]: [INFO] [05-31|01:14:43.177] head updated                             hash=0xa2a051d3a8e0ae5edeedd59e3fa5c350c8cd676cf773dbb09d45a74d3d72e696 number=19985565
may 31 01:14:54 root-pc erigon[4745]: [INFO] [05-31|01:14:54.849] [p2p] GoodPeers                          eth68=6
may 31 01:14:55 root-pc erigon[4745]: [INFO] [05-31|01:14:55.654] [mem] memory stats                       Rss=18.6MB Size=0B Pss=18.6MB SharedClean=2.9KB SharedDirty=0B PrivateClean=6.0MB PrivateDirty=12.6MB Referenced=18.6MB Anonymous=12.5MB Swap=0B alloc=10.6GB sys=12.3GB
may 31 01:14:55 root-pc erigon[4745]: [INFO] [05-31|01:14:55.716] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
may 31 01:14:56 root-pc erigon[4745]: [INFO] [05-31|01:14:56.236] RPC Daemon notified of new headers       from=19985565 to=19985566 amount=1 hash=0xd911afcedc1e3be0a0a32ae5d2305b5bd6b6dc361a5f18ebafa6e82ad2505dc4 header sending=23.164µs log sending=413ns
may 31 01:14:56 root-pc erigon[4745]: [INFO] [05-31|01:14:56.236] head updated                             hash=0xd911afcedc1e3be0a0a32ae5d2305b5bd6b6dc361a5f18ebafa6e82ad2505dc4 number=19985566
may 31 01:14:56 root-pc erigon[4745]: [INFO] [05-31|01:14:56.306] [txpool] stat                            pending=91 baseFee=214 queued=3 alloc=10.8GB sys=12.4GB
may 31 01:15:07 root-pc erigon[4745]: [INFO] [05-31|01:15:07.402] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
may 31 01:15:07 root-pc erigon[4745]: [INFO] [05-31|01:15:07.878] RPC Daemon notified of new headers       from=19985566 to=19985567 amount=1 hash=0x14c13d6068a1b7ae2d3adcc6624579b57e579e0b7f4dad9b8763b61c5e27d9ad header sending=67.019µs log sending=508ns
may 31 01:15:07 root-pc erigon[4745]: [INFO] [05-31|01:15:07.878] head updated                             hash=0x14c13d6068a1b7ae2d3adcc6624579b57e579e0b7f4dad9b8763b61c5e27d9ad number=19985567
may 31 01:15:25 root-pc erigon[4745]: [INFO] [05-31|01:15:25.461] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
may 31 01:15:25 root-pc erigon[4745]: [INFO] [05-31|01:15:25.976] RPC Daemon notified of new headers       from=19985567 to=19985568 amount=1 hash=0x62754756487aecbc381e3f9d16a03455d805630acf6a80af9095f7f79e07904b header sending=30.216µs log sending=1.53µs
may 31 01:15:25 root-pc erigon[4745]: [INFO] [05-31|01:15:25.976] head updated                             hash=0x62754756487aecbc381e3f9d16a03455d805630acf6a80af9095f7f79e07904b number=19985568
may 31 01:15:32 root-pc erigon[4745]: [INFO] [05-31|01:15:32.851] P2P                                      app=caplin peers=67
may 31 01:16:22 root-pc erigon[4745]: [INFO] [05-31|01:16:22.014] [4/12 Execution] Completed on            block=19985573
may 31 01:16:23 root-pc erigon[4745]: [INFO] [05-31|01:16:23.664] RPC Daemon notified of new headers       from=19985568 to=19985573 amount=5 hash=0x6ce6cd0986fa51fd34c942af9c71922d875da60f09a1a9135be5f36fb5333fe2 header sending=25.208µs log sending=412ns
may 31 01:16:23 root-pc erigon[4745]: [INFO] [05-31|01:16:23.664] head updated                             hash=0x6ce6cd0986fa51fd34c942af9c71922d875da60f09a1a9135be5f36fb5333fe2 number=19985573

AskAlexSharov commented 4 months ago

Let's do next steps:

Add —no-downloader=true —snap.stop=true
Add WRITE_MAP=true MDBX_DIRTY_SPACE_MB=1024
start and let erigon work ~30min (several sync cycles). When see issue again: do curl http://127.0.0.1:6060/debug/metrics/prometheus show here

fridary commented 4 months ago

@AskAlexSharov the same. 5-50 seconds delay. I waited 10 hours. curl http://127.0.0.1:6060/debug/metrics/prometheus:

# HELP beacon_process_attestation_post_altair
# TYPE beacon_process_attestation_post_altair histogram
beacon_process_attestation_post_altair_bucket{le="0.005"} 0
beacon_process_attestation_post_altair_bucket{le="0.01"} 0
beacon_process_attestation_post_altair_bucket{le="0.025"} 0
beacon_process_attestation_post_altair_bucket{le="0.05"} 0
beacon_process_attestation_post_altair_bucket{le="0.1"} 0
beacon_process_attestation_post_altair_bucket{le="0.25"} 0
beacon_process_attestation_post_altair_bucket{le="0.5"} 0
beacon_process_attestation_post_altair_bucket{le="1"} 0
beacon_process_attestation_post_altair_bucket{le="2.5"} 0
beacon_process_attestation_post_altair_bucket{le="5"} 0
beacon_process_attestation_post_altair_bucket{le="10"} 0
beacon_process_attestation_post_altair_bucket{le="+Inf"} 0
beacon_process_attestation_post_altair_sum 0
beacon_process_attestation_post_altair_count 0
beacon_process_attestation_post_altair_bucket{step="get_attesting_indices",le="0.005"} 394546
beacon_process_attestation_post_altair_bucket{step="get_attesting_indices",le="0.01"} 394547
beacon_process_attestation_post_altair_bucket{step="get_attesting_indices",le="0.025"} 394547
beacon_process_attestation_post_altair_bucket{step="get_attesting_indices",le="0.05"} 394547
beacon_process_attestation_post_altair_bucket{step="get_attesting_indices",le="0.1"} 394547
beacon_process_attestation_post_altair_bucket{step="get_attesting_indices",le="0.25"} 394547
beacon_process_attestation_post_altair_bucket{step="get_attesting_indices",le="0.5"} 394636
beacon_process_attestation_post_altair_bucket{step="get_attesting_indices",le="1"} 394636
beacon_process_attestation_post_altair_bucket{step="get_attesting_indices",le="2.5"} 394636
beacon_process_attestation_post_altair_bucket{step="get_attesting_indices",le="5"} 394636
beacon_process_attestation_post_altair_bucket{step="get_attesting_indices",le="10"} 394636
beacon_process_attestation_post_altair_bucket{step="get_attesting_indices",le="+Inf"} 394636
beacon_process_attestation_post_altair_sum{step="get_attesting_indices"} 36.1882251879987
beacon_process_attestation_post_altair_count{step="get_attesting_indices"} 394636
beacon_process_attestation_post_altair_bucket{step="get_participation_flag",le="0.005"} 394636
beacon_process_attestation_post_altair_bucket{step="get_participation_flag",le="0.01"} 394636
beacon_process_attestation_post_altair_bucket{step="get_participation_flag",le="0.025"} 394636
beacon_process_attestation_post_altair_bucket{step="get_participation_flag",le="0.05"} 394636
beacon_process_attestation_post_altair_bucket{step="get_participation_flag",le="0.1"} 394636
beacon_process_attestation_post_altair_bucket{step="get_participation_flag",le="0.25"} 394636
beacon_process_attestation_post_altair_bucket{step="get_participation_flag",le="0.5"} 394636
beacon_process_attestation_post_altair_bucket{step="get_participation_flag",le="1"} 394636
beacon_process_attestation_post_altair_bucket{step="get_participation_flag",le="2.5"} 394636
beacon_process_attestation_post_altair_bucket{step="get_participation_flag",le="5"} 394636
beacon_process_attestation_post_altair_bucket{step="get_participation_flag",le="10"} 394636
beacon_process_attestation_post_altair_bucket{step="get_participation_flag",le="+Inf"} 394636
beacon_process_attestation_post_altair_sum{step="get_participation_flag"} 0.2618031790000049
beacon_process_attestation_post_altair_count{step="get_participation_flag"} 394636
beacon_process_attestation_post_altair_bucket{step="get_proposer_index",le="0.005"} 394636
beacon_process_attestation_post_altair_bucket{step="get_proposer_index",le="0.01"} 394636
beacon_process_attestation_post_altair_bucket{step="get_proposer_index",le="0.025"} 394636
beacon_process_attestation_post_altair_bucket{step="get_proposer_index",le="0.05"} 394636
beacon_process_attestation_post_altair_bucket{step="get_proposer_index",le="0.1"} 394636
beacon_process_attestation_post_altair_bucket{step="get_proposer_index",le="0.25"} 394636
beacon_process_attestation_post_altair_bucket{step="get_proposer_index",le="0.5"} 394636
beacon_process_attestation_post_altair_bucket{step="get_proposer_index",le="1"} 394636
beacon_process_attestation_post_altair_bucket{step="get_proposer_index",le="2.5"} 394636
beacon_process_attestation_post_altair_bucket{step="get_proposer_index",le="5"} 394636
beacon_process_attestation_post_altair_bucket{step="get_proposer_index",le="10"} 394636
beacon_process_attestation_post_altair_bucket{step="get_proposer_index",le="+Inf"} 394636
beacon_process_attestation_post_altair_sum{step="get_proposer_index"} 0.08197017500001057
beacon_process_attestation_post_altair_count{step="get_proposer_index"} 394636
beacon_process_attestation_post_altair_bucket{step="update_attestation",le="0.005"} 394635
beacon_process_attestation_post_altair_bucket{step="update_attestation",le="0.01"} 394636
beacon_process_attestation_post_altair_bucket{step="update_attestation",le="0.025"} 394636
beacon_process_attestation_post_altair_bucket{step="update_attestation",le="0.05"} 394636
beacon_process_attestation_post_altair_bucket{step="update_attestation",le="0.1"} 394636
beacon_process_attestation_post_altair_bucket{step="update_attestation",le="0.25"} 394636
beacon_process_attestation_post_altair_bucket{step="update_attestation",le="0.5"} 394636
beacon_process_attestation_post_altair_bucket{step="update_attestation",le="1"} 394636
beacon_process_attestation_post_altair_bucket{step="update_attestation",le="2.5"} 394636
beacon_process_attestation_post_altair_bucket{step="update_attestation",le="5"} 394636
beacon_process_attestation_post_altair_bucket{step="update_attestation",le="10"} 394636
beacon_process_attestation_post_altair_bucket{step="update_attestation",le="+Inf"} 394636
beacon_process_attestation_post_altair_sum{step="update_attestation"} 35.012018179000485
beacon_process_attestation_post_altair_count{step="update_attestation"} 394636
# HELP beacon_process_attestations
# TYPE beacon_process_attestations histogram
beacon_process_attestations_bucket{le="0.005"} 0
beacon_process_attestations_bucket{le="0.01"} 0
beacon_process_attestations_bucket{le="0.025"} 0
beacon_process_attestations_bucket{le="0.05"} 0
beacon_process_attestations_bucket{le="0.1"} 0
beacon_process_attestations_bucket{le="0.25"} 0
beacon_process_attestations_bucket{le="0.5"} 0
beacon_process_attestations_bucket{le="1"} 0
beacon_process_attestations_bucket{le="2.5"} 0
beacon_process_attestations_bucket{le="5"} 0
beacon_process_attestations_bucket{le="10"} 0
beacon_process_attestations_bucket{le="+Inf"} 0
beacon_process_attestations_sum 0
beacon_process_attestations_count 0
beacon_process_attestations_bucket{attestation_step="process",le="0.005"} 0
beacon_process_attestations_bucket{attestation_step="process",le="0.01"} 214
beacon_process_attestations_bucket{attestation_step="process",le="0.025"} 3335
beacon_process_attestations_bucket{attestation_step="process",le="0.05"} 3364
beacon_process_attestations_bucket{attestation_step="process",le="0.1"} 3371
beacon_process_attestations_bucket{attestation_step="process",le="0.25"} 3371
beacon_process_attestations_bucket{attestation_step="process",le="0.5"} 3458
beacon_process_attestations_bucket{attestation_step="process",le="1"} 3459
beacon_process_attestations_bucket{attestation_step="process",le="2.5"} 3459
beacon_process_attestations_bucket{attestation_step="process",le="5"} 3459
beacon_process_attestations_bucket{attestation_step="process",le="10"} 3459
beacon_process_attestations_bucket{attestation_step="process",le="+Inf"} 3459
beacon_process_attestations_sum{attestation_step="process"} 74.26662826099988
beacon_process_attestations_count{attestation_step="process"} 3459
beacon_process_attestations_bucket{attestation_step="validate",le="0.005"} 0
beacon_process_attestations_bucket{attestation_step="validate",le="0.01"} 0
beacon_process_attestations_bucket{attestation_step="validate",le="0.025"} 0
beacon_process_attestations_bucket{attestation_step="validate",le="0.05"} 6
beacon_process_attestations_bucket{attestation_step="validate",le="0.1"} 2713
beacon_process_attestations_bucket{attestation_step="validate",le="0.25"} 2775
beacon_process_attestations_bucket{attestation_step="validate",le="0.5"} 2776
beacon_process_attestations_bucket{attestation_step="validate",le="1"} 2776
beacon_process_attestations_bucket{attestation_step="validate",le="2.5"} 2776
beacon_process_attestations_bucket{attestation_step="validate",le="5"} 2776
beacon_process_attestations_bucket{attestation_step="validate",le="10"} 2776
beacon_process_attestations_bucket{attestation_step="validate",le="+Inf"} 2776
beacon_process_attestations_sum{attestation_step="validate"} 185.31452778300041
beacon_process_attestations_count{attestation_step="validate"} 2776
# HELP beacon_process_block
# TYPE beacon_process_block histogram
beacon_process_block_bucket{le="0.005"} 0
beacon_process_block_bucket{le="0.01"} 38
beacon_process_block_bucket{le="0.025"} 661
beacon_process_block_bucket{le="0.05"} 664
beacon_process_block_bucket{le="0.1"} 3042
beacon_process_block_bucket{le="0.25"} 3367
beacon_process_block_bucket{le="0.5"} 3453
beacon_process_block_bucket{le="1"} 3459
beacon_process_block_bucket{le="2.5"} 3459
beacon_process_block_bucket{le="5"} 3459
beacon_process_block_bucket{le="10"} 3459
beacon_process_block_bucket{le="+Inf"} 3459
beacon_process_block_sum 286.72559088899965
beacon_process_block_count 3459
# HELP block_consumer_delay
# TYPE block_consumer_delay summary
block_consumer_delay{type="body_download",quantile="0.5"} 8.46951703
block_consumer_delay{type="body_download",quantile="0.9"} 18.172671999
block_consumer_delay{type="body_download",quantile="0.97"} 21.708001487
block_consumer_delay{type="body_download",quantile="0.99"} 21.906274566
block_consumer_delay_sum{type="body_download"} 318709.0456862282
block_consumer_delay_count{type="body_download"} 5628
block_consumer_delay{type="header_download",quantile="0.5"} 8.469509384
block_consumer_delay{type="header_download",quantile="0.9"} 18.172671189
block_consumer_delay{type="header_download",quantile="0.97"} 21.707992357
block_consumer_delay{type="header_download",quantile="0.99"} 21.906272169
block_consumer_delay_sum{type="header_download"} 318709.0345189774
block_consumer_delay_count{type="header_download"} 5628
block_consumer_delay{type="post_execution",quantile="0.5"} 27.843971689
block_consumer_delay{type="post_execution",quantile="0.9"} 141.128283732
block_consumer_delay{type="post_execution",quantile="0.97"} 168.041626433
block_consumer_delay{type="post_execution",quantile="0.99"} 179.986337802
block_consumer_delay_sum{type="post_execution"} 1.0460630447368964e+06
block_consumer_delay_count{type="post_execution"} 10796
block_consumer_delay{type="pre_execution",quantile="0.5"} 27.778172165
block_consumer_delay{type="pre_execution",quantile="0.9"} 140.991587028
block_consumer_delay{type="pre_execution",quantile="0.97"} 167.986878532
block_consumer_delay{type="pre_execution",quantile="0.99"} 179.907383515
block_consumer_delay_sum{type="pre_execution"} 1.044979869506132e+06
block_consumer_delay_count{type="pre_execution"} 10796
# HELP block_producer_delay
# TYPE block_producer_delay summary
block_producer_delay{type="production",quantile="0.5"} NaN
block_producer_delay{type="production",quantile="0.9"} NaN
block_producer_delay{type="production",quantile="0.97"} NaN
block_producer_delay{type="production",quantile="0.99"} NaN
block_producer_delay_sum{type="production"} 0
block_producer_delay_count{type="production"} 0
# HELP cache_total
# TYPE cache_total counter
cache_total{target="acc_read"} 0
cache_total{target="st_read"} 0
cache_total{target="write"} 0
# HELP chain_autorewind_length
# TYPE chain_autorewind_length gauge
chain_autorewind_length 0
# HELP chain_checkpoint_isvalidchain
# TYPE chain_checkpoint_isvalidchain gauge
chain_checkpoint_isvalidchain 0
# HELP chain_checkpoint_latest
# TYPE chain_checkpoint_latest gauge
chain_checkpoint_latest 0
# HELP chain_execution_seconds
# TYPE chain_execution_seconds summary
chain_execution_seconds{quantile="0.5"} 0.083827948
chain_execution_seconds{quantile="0.9"} 0.185247641
chain_execution_seconds{quantile="0.97"} 0.26229192
chain_execution_seconds{quantile="0.99"} 0.288277635
chain_execution_seconds_sum 1067.4500650879997
chain_execution_seconds_count 10796
# HELP chain_milestone_future
# TYPE chain_milestone_future gauge
chain_milestone_future 0
# HELP chain_milestone_idslength
# TYPE chain_milestone_idslength gauge
chain_milestone_idslength 0
# HELP chain_milestone_isvalidchain
# TYPE chain_milestone_isvalidchain gauge
chain_milestone_isvalidchain 0
# HELP chain_milestone_latest
# TYPE chain_milestone_latest gauge
chain_milestone_latest 0
# HELP client_requests_checkpoint_duration
# TYPE client_requests_checkpoint_duration summary
client_requests_checkpoint_duration{quantile="0.5"} NaN
client_requests_checkpoint_duration{quantile="0.9"} NaN
client_requests_checkpoint_duration{quantile="0.97"} NaN
client_requests_checkpoint_duration{quantile="0.99"} NaN
client_requests_checkpoint_duration_sum 0
client_requests_checkpoint_duration_count 0
# HELP client_requests_checkpoint_invalid
# TYPE client_requests_checkpoint_invalid gauge
client_requests_checkpoint_invalid 0
# HELP client_requests_checkpoint_valid
# TYPE client_requests_checkpoint_valid gauge
client_requests_checkpoint_valid 0
# HELP client_requests_checkpointcount_duration
# TYPE client_requests_checkpointcount_duration summary
client_requests_checkpointcount_duration{quantile="0.5"} NaN
client_requests_checkpointcount_duration{quantile="0.9"} NaN
client_requests_checkpointcount_duration{quantile="0.97"} NaN
client_requests_checkpointcount_duration{quantile="0.99"} NaN
client_requests_checkpointcount_duration_sum 0
client_requests_checkpointcount_duration_count 0
# HELP client_requests_checkpointcount_invalid
# TYPE client_requests_checkpointcount_invalid gauge
client_requests_checkpointcount_invalid 0
# HELP client_requests_checkpointcount_valid
# TYPE client_requests_checkpointcount_valid gauge
client_requests_checkpointcount_valid 0
# HELP client_requests_span_duration
# TYPE client_requests_span_duration summary
client_requests_span_duration{quantile="0.5"} NaN
client_requests_span_duration{quantile="0.9"} NaN
client_requests_span_duration{quantile="0.97"} NaN
client_requests_span_duration{quantile="0.99"} NaN
client_requests_span_duration_sum 0
client_requests_span_duration_count 0
# HELP client_requests_span_invalid
# TYPE client_requests_span_invalid gauge
client_requests_span_invalid 0
# HELP client_requests_span_valid
# TYPE client_requests_span_valid gauge
client_requests_span_valid 0
# HELP client_requests_statesync_duration
# TYPE client_requests_statesync_duration summary
client_requests_statesync_duration{quantile="0.5"} NaN
client_requests_statesync_duration{quantile="0.9"} NaN
client_requests_statesync_duration{quantile="0.97"} NaN
client_requests_statesync_duration{quantile="0.99"} NaN
client_requests_statesync_duration_sum 0
client_requests_statesync_duration_count 0
# HELP client_requests_statesync_invalid
# TYPE client_requests_statesync_invalid gauge
client_requests_statesync_invalid 0
# HELP client_requests_statesync_valid
# TYPE client_requests_statesync_valid gauge
client_requests_statesync_valid 0
# HELP db_commit_seconds
# TYPE db_commit_seconds summary
db_commit_seconds{phase="ending",quantile="0.5"} 1.5258e-05
db_commit_seconds{phase="ending",quantile="0.9"} 0.00015258
db_commit_seconds{phase="ending",quantile="0.97"} 0.002120862
db_commit_seconds{phase="ending",quantile="0.99"} 0.004668948
db_commit_seconds_sum{phase="ending"} 4.138183211999816
db_commit_seconds_count{phase="ending"} 10884
db_commit_seconds{phase="preparation",quantile="0.5"} 1.5258e-05
db_commit_seconds{phase="preparation",quantile="0.9"} 1.5258e-05
db_commit_seconds{phase="preparation",quantile="0.97"} 1.5258e-05
db_commit_seconds{phase="preparation",quantile="0.99"} 1.5258e-05
db_commit_seconds_sum{phase="preparation"} 0.08518541400000113
db_commit_seconds_count{phase="preparation"} 10884
db_commit_seconds{phase="sync",quantile="0.5"} 4.5774e-05
db_commit_seconds{phase="sync",quantile="0.9"} 0.052365456
db_commit_seconds{phase="sync",quantile="0.97"} 0.15235113
db_commit_seconds{phase="sync",quantile="0.99"} 0.435554868
db_commit_seconds_sum{phase="sync"} 112.98773292600207
db_commit_seconds_count{phase="sync"} 10884
db_commit_seconds{phase="total",quantile="0.5"} 0.000778158
db_commit_seconds{phase="total",quantile="0.9"} 0.188756718
db_commit_seconds{phase="total",quantile="0.97"} 0.25984374
db_commit_seconds{phase="total",quantile="0.99"} 0.435814254
db_commit_seconds_sum{phase="total"} 380.1049615260226
db_commit_seconds_count{phase="total"} 10884
db_commit_seconds{phase="write",quantile="0.5"} 9.1548e-05
db_commit_seconds{phase="write",quantile="0.9"} 0.116082864
db_commit_seconds{phase="write",quantile="0.97"} 0.17958666
db_commit_seconds{phase="write",quantile="0.99"} 0.203587494
db_commit_seconds_sum{phase="write"} 177.8435708220029
db_commit_seconds_count{phase="write"} 10884
# HELP db_gc_leaf
# TYPE db_gc_leaf gauge
db_gc_leaf 399
# HELP db_gc_overflow
# TYPE db_gc_overflow gauge
db_gc_overflow 12240
# HELP db_gc_pages
# TYPE db_gc_pages gauge
db_gc_pages 1.2942336e+07
# HELP db_pgops
# TYPE db_pgops gauge
db_pgops{phase="clone"} 0
db_pgops{phase="cow"} 2.0564158e+07
db_pgops{phase="merge"} 638
db_pgops{phase="newly"} 302261
db_pgops{phase="spill"} 5.393686e+06
db_pgops{phase="split"} 85195
db_pgops{phase="unspill"} 258
db_pgops{phase="wops"} 2.0883663e+07
# HELP db_size
# TYPE db_size gauge
db_size 1.480572534784e+12
# HELP decode_ssz_beacon_state_dur
# TYPE decode_ssz_beacon_state_dur histogram
decode_ssz_beacon_state_dur_bucket{le="0.005"} 1
decode_ssz_beacon_state_dur_bucket{le="0.01"} 1
decode_ssz_beacon_state_dur_bucket{le="0.025"} 1
decode_ssz_beacon_state_dur_bucket{le="0.05"} 1
decode_ssz_beacon_state_dur_bucket{le="0.1"} 19
decode_ssz_beacon_state_dur_bucket{le="0.25"} 417
decode_ssz_beacon_state_dur_bucket{le="0.5"} 450
decode_ssz_beacon_state_dur_bucket{le="1"} 451
decode_ssz_beacon_state_dur_bucket{le="2.5"} 451
decode_ssz_beacon_state_dur_bucket{le="5"} 451
decode_ssz_beacon_state_dur_bucket{le="10"} 451
decode_ssz_beacon_state_dur_bucket{le="+Inf"} 451
decode_ssz_beacon_state_dur_sum 68.56626919299998
decode_ssz_beacon_state_dur_count 451
# HELP decode_ssz_beacon_state_size
# TYPE decode_ssz_beacon_state_size histogram
decode_ssz_beacon_state_size_bucket{le="0.005"} 0
decode_ssz_beacon_state_size_bucket{le="0.01"} 0
decode_ssz_beacon_state_size_bucket{le="0.025"} 0
decode_ssz_beacon_state_size_bucket{le="0.05"} 0
decode_ssz_beacon_state_size_bucket{le="0.1"} 0
decode_ssz_beacon_state_size_bucket{le="0.25"} 0
decode_ssz_beacon_state_size_bucket{le="0.5"} 0
decode_ssz_beacon_state_size_bucket{le="1"} 0
decode_ssz_beacon_state_size_bucket{le="2.5"} 0
decode_ssz_beacon_state_size_bucket{le="5"} 0
decode_ssz_beacon_state_size_bucket{le="10"} 0
decode_ssz_beacon_state_size_bucket{le="+Inf"} 451
decode_ssz_beacon_state_size_sum 9.0103837561e+10
decode_ssz_beacon_state_size_count 451
# HELP domain_block_current
# TYPE domain_block_current gauge
domain_block_current 0
# HELP domain_build_files_took
# TYPE domain_build_files_took summary
domain_build_files_took{quantile="0.5"} NaN
domain_build_files_took{quantile="0.9"} NaN
domain_build_files_took{quantile="0.97"} NaN
domain_build_files_took{quantile="0.99"} NaN
domain_build_files_took_sum 0
domain_build_files_took_count 0
# HELP domain_collate_took
# TYPE domain_collate_took histogram
domain_collate_took_bucket{le="0.005"} 0
domain_collate_took_bucket{le="0.01"} 0
domain_collate_took_bucket{le="0.025"} 0
domain_collate_took_bucket{le="0.05"} 0
domain_collate_took_bucket{le="0.1"} 0
domain_collate_took_bucket{le="0.25"} 0
domain_collate_took_bucket{le="0.5"} 0
domain_collate_took_bucket{le="1"} 0
domain_collate_took_bucket{le="2.5"} 0
domain_collate_took_bucket{le="5"} 0
domain_collate_took_bucket{le="10"} 0
domain_collate_took_bucket{le="+Inf"} 0
domain_collate_took_sum 0
domain_collate_took_count 0
# HELP domain_collation_hist_size
# TYPE domain_collation_hist_size gauge
domain_collation_hist_size 0
# HELP domain_collation_size
# TYPE domain_collation_size gauge
domain_collation_size 0
# HELP domain_commitment_keys
# TYPE domain_commitment_keys counter
domain_commitment_keys 0
# HELP domain_commitment_took
# TYPE domain_commitment_took summary
domain_commitment_took{quantile="0.5"} NaN
domain_commitment_took{quantile="0.9"} NaN
domain_commitment_took{quantile="0.97"} NaN
domain_commitment_took{quantile="0.99"} NaN
domain_commitment_took_sum 0
domain_commitment_took_count 0
# HELP domain_commitment_updates
# TYPE domain_commitment_updates counter
domain_commitment_updates 0
# HELP domain_commitment_updates_applied
# TYPE domain_commitment_updates_applied counter
domain_commitment_updates_applied 0
# HELP domain_commitment_write_took
# TYPE domain_commitment_write_took histogram
domain_commitment_write_took_bucket{le="0.005"} 0
domain_commitment_write_took_bucket{le="0.01"} 0
domain_commitment_write_took_bucket{le="0.025"} 0
domain_commitment_write_took_bucket{le="0.05"} 0
domain_commitment_write_took_bucket{le="0.1"} 0
domain_commitment_write_took_bucket{le="0.25"} 0
domain_commitment_write_took_bucket{le="0.5"} 0
domain_commitment_write_took_bucket{le="1"} 0
domain_commitment_write_took_bucket{le="2.5"} 0
domain_commitment_write_took_bucket{le="5"} 0
domain_commitment_write_took_bucket{le="10"} 0
domain_commitment_write_took_bucket{le="+Inf"} 0
domain_commitment_write_took_sum 0
domain_commitment_write_took_count 0
# HELP domain_prune_hist_took
# TYPE domain_prune_hist_took histogram
domain_prune_hist_took_bucket{le="0.005"} 0
domain_prune_hist_took_bucket{le="0.01"} 0
domain_prune_hist_took_bucket{le="0.025"} 0
domain_prune_hist_took_bucket{le="0.05"} 0
domain_prune_hist_took_bucket{le="0.1"} 0
domain_prune_hist_took_bucket{le="0.25"} 0
domain_prune_hist_took_bucket{le="0.5"} 0
domain_prune_hist_took_bucket{le="1"} 0
domain_prune_hist_took_bucket{le="2.5"} 0
domain_prune_hist_took_bucket{le="5"} 0
domain_prune_hist_took_bucket{le="10"} 0
domain_prune_hist_took_bucket{le="+Inf"} 0
domain_prune_hist_took_sum 0
domain_prune_hist_took_count 0
# HELP domain_prune_size
# TYPE domain_prune_size counter
domain_prune_size 0
# HELP domain_prune_took
# TYPE domain_prune_took histogram
domain_prune_took_bucket{le="0.005"} 0
domain_prune_took_bucket{le="0.01"} 0
domain_prune_took_bucket{le="0.025"} 0
domain_prune_took_bucket{le="0.05"} 0
domain_prune_took_bucket{le="0.1"} 0
domain_prune_took_bucket{le="0.25"} 0
domain_prune_took_bucket{le="0.5"} 0
domain_prune_took_bucket{le="1"} 0
domain_prune_took_bucket{le="2.5"} 0
domain_prune_took_bucket{le="5"} 0
domain_prune_took_bucket{le="10"} 0
domain_prune_took_bucket{le="+Inf"} 0
domain_prune_took_sum 0
domain_prune_took_count 0
# HELP domain_pruning_progress
# TYPE domain_pruning_progress gauge
domain_pruning_progress 0
# HELP domain_running_collations
# TYPE domain_running_collations gauge
domain_running_collations 0
# HELP domain_running_commitment
# TYPE domain_running_commitment gauge
domain_running_commitment 0
# HELP domain_running_merges
# TYPE domain_running_merges gauge
domain_running_merges 0
# HELP domain_step_current
# TYPE domain_step_current gauge
domain_step_current 0
# HELP domain_step_took
# TYPE domain_step_took histogram
domain_step_took_bucket{le="0.005"} 0
domain_step_took_bucket{le="0.01"} 0
domain_step_took_bucket{le="0.025"} 0
domain_step_took_bucket{le="0.05"} 0
domain_step_took_bucket{le="0.1"} 0
domain_step_took_bucket{le="0.25"} 0
domain_step_took_bucket{le="0.5"} 0
domain_step_took_bucket{le="1"} 0
domain_step_took_bucket{le="2.5"} 0
domain_step_took_bucket{le="5"} 0
domain_step_took_bucket{le="10"} 0
domain_step_took_bucket{le="+Inf"} 0
domain_step_took_sum 0
domain_step_took_count 0
# HELP domain_tx_processed
# TYPE domain_tx_processed gauge
domain_tx_processed 0
# HELP encode_ssz_beacon_state_dur
# TYPE encode_ssz_beacon_state_dur histogram
encode_ssz_beacon_state_dur_bucket{le="0.005"} 0
encode_ssz_beacon_state_dur_bucket{le="0.01"} 0
encode_ssz_beacon_state_dur_bucket{le="0.025"} 0
encode_ssz_beacon_state_dur_bucket{le="0.05"} 615
encode_ssz_beacon_state_dur_bucket{le="0.1"} 670
encode_ssz_beacon_state_dur_bucket{le="0.25"} 687
encode_ssz_beacon_state_dur_bucket{le="0.5"} 692
encode_ssz_beacon_state_dur_bucket{le="1"} 692
encode_ssz_beacon_state_dur_bucket{le="2.5"} 692
encode_ssz_beacon_state_dur_bucket{le="5"} 692
encode_ssz_beacon_state_dur_bucket{le="10"} 692
encode_ssz_beacon_state_dur_bucket{le="+Inf"} 692
encode_ssz_beacon_state_dur_sum 30.605962034000022
encode_ssz_beacon_state_dur_count 692
# HELP encode_ssz_beacon_state_size
# TYPE encode_ssz_beacon_state_size histogram
encode_ssz_beacon_state_size_bucket{le="0.005"} 0
encode_ssz_beacon_state_size_bucket{le="0.01"} 0
encode_ssz_beacon_state_size_bucket{le="0.025"} 0
encode_ssz_beacon_state_size_bucket{le="0.05"} 0
encode_ssz_beacon_state_size_bucket{le="0.1"} 0
encode_ssz_beacon_state_size_bucket{le="0.25"} 0
encode_ssz_beacon_state_size_bucket{le="0.5"} 0
encode_ssz_beacon_state_size_bucket{le="1"} 0
encode_ssz_beacon_state_size_bucket{le="2.5"} 0
encode_ssz_beacon_state_size_bucket{le="5"} 0
encode_ssz_beacon_state_size_bucket{le="10"} 0
encode_ssz_beacon_state_size_bucket{le="+Inf"} 692
encode_ssz_beacon_state_size_sum 1.38549970605e+11
encode_ssz_beacon_state_size_count 692
# HELP erigon_info
# TYPE erigon_info gauge
erigon_info{commit="088fd8ef69389a72da6faa0fc7903a4ba5726911",version="2.59.3"} 1
# HELP exec_repeats
# TYPE exec_repeats counter
exec_repeats 0
# HELP exec_steps_in_db
# TYPE exec_steps_in_db gauge
exec_steps_in_db 0
# HELP exec_triggers
# TYPE exec_triggers counter
exec_triggers 0
# HELP exec_txs_done
# TYPE exec_txs_done counter
exec_txs_done 0
# HELP go_gc_duration_seconds A summary of the pause duration of garbage collection cycles.
# TYPE go_gc_duration_seconds summary
go_gc_duration_seconds{quantile="0"} 0.000244401
go_gc_duration_seconds{quantile="0.25"} 0.000414059
go_gc_duration_seconds{quantile="0.5"} 0.000544551
go_gc_duration_seconds{quantile="0.75"} 0.00075779
go_gc_duration_seconds{quantile="1"} 0.038133647
go_gc_duration_seconds_sum 0.368497262
go_gc_duration_seconds_count 375
# HELP go_goroutines Number of goroutines that currently exist.
# TYPE go_goroutines gauge
go_goroutines 9779
# HELP go_info Information about the Go environment.
# TYPE go_info gauge
go_info{version="go1.22.1"} 1
# HELP go_memstats_alloc_bytes Number of bytes allocated and still in use.
# TYPE go_memstats_alloc_bytes gauge
go_memstats_alloc_bytes 1.3523471776e+10
# HELP go_memstats_alloc_bytes_total Total number of bytes allocated, even if freed.
# TYPE go_memstats_alloc_bytes_total counter
go_memstats_alloc_bytes_total 2.251429501904e+12
# HELP go_memstats_buck_hash_sys_bytes Number of bytes used by the profiling bucket hash table.
# TYPE go_memstats_buck_hash_sys_bytes gauge
go_memstats_buck_hash_sys_bytes 1.9631645e+07
# HELP go_memstats_frees_total Total number of frees.
# TYPE go_memstats_frees_total counter
go_memstats_frees_total 5.586389989e+09
# HELP go_memstats_gc_sys_bytes Number of bytes used for garbage collection system metadata.
# TYPE go_memstats_gc_sys_bytes gauge
go_memstats_gc_sys_bytes 8.0495736e+07
# HELP go_memstats_heap_alloc_bytes Number of heap bytes allocated and still in use.
# TYPE go_memstats_heap_alloc_bytes gauge
go_memstats_heap_alloc_bytes 1.3523471776e+10
# HELP go_memstats_heap_idle_bytes Number of heap bytes waiting to be used.
# TYPE go_memstats_heap_idle_bytes gauge
go_memstats_heap_idle_bytes 5.738504192e+09
# HELP go_memstats_heap_inuse_bytes Number of heap bytes that are in use.
# TYPE go_memstats_heap_inuse_bytes gauge
go_memstats_heap_inuse_bytes 1.3680246784e+10
# HELP go_memstats_heap_objects Number of allocated objects.
# TYPE go_memstats_heap_objects gauge
go_memstats_heap_objects 4.9870015e+07
# HELP go_memstats_heap_released_bytes Number of heap bytes released to OS.
# TYPE go_memstats_heap_released_bytes gauge
go_memstats_heap_released_bytes 4.03283968e+09
# HELP go_memstats_heap_sys_bytes Number of heap bytes obtained from system.
# TYPE go_memstats_heap_sys_bytes gauge
go_memstats_heap_sys_bytes 1.9418750976e+10
# HELP go_memstats_last_gc_time_seconds Number of seconds since 1970 of last garbage collection.
# TYPE go_memstats_last_gc_time_seconds gauge
go_memstats_last_gc_time_seconds 1.7171576251517787e+09
# HELP go_memstats_lookups_total Total number of pointer lookups.
# TYPE go_memstats_lookups_total counter
go_memstats_lookups_total 0
# HELP go_memstats_mallocs_total Total number of mallocs.
# TYPE go_memstats_mallocs_total counter
go_memstats_mallocs_total 5.636260004e+09
# HELP go_memstats_mcache_inuse_bytes Number of bytes in use by mcache structures.
# TYPE go_memstats_mcache_inuse_bytes gauge
go_memstats_mcache_inuse_bytes 86400
# HELP go_memstats_mcache_sys_bytes Number of bytes used for mcache structures obtained from system.
# TYPE go_memstats_mcache_sys_bytes gauge
go_memstats_mcache_sys_bytes 93600
# HELP go_memstats_mspan_inuse_bytes Number of bytes in use by mspan structures.
# TYPE go_memstats_mspan_inuse_bytes gauge
go_memstats_mspan_inuse_bytes 9.151376e+07
# HELP go_memstats_mspan_sys_bytes Number of bytes used for mspan structures obtained from system.
# TYPE go_memstats_mspan_sys_bytes gauge
go_memstats_mspan_sys_bytes 1.0146144e+08
# HELP go_memstats_next_gc_bytes Number of heap bytes when next garbage collection will take place.
# TYPE go_memstats_next_gc_bytes gauge
go_memstats_next_gc_bytes 1.3934945288e+10
# HELP go_memstats_other_sys_bytes Number of bytes used for other system allocations.
# TYPE go_memstats_other_sys_bytes gauge
go_memstats_other_sys_bytes 2.0848787e+07
# HELP go_memstats_stack_inuse_bytes Number of bytes in use by the stack allocator.
# TYPE go_memstats_stack_inuse_bytes gauge
go_memstats_stack_inuse_bytes 9.0210304e+07
# HELP go_memstats_stack_sys_bytes Number of bytes obtained from system for stack allocator.
# TYPE go_memstats_stack_sys_bytes gauge
go_memstats_stack_sys_bytes 9.0210304e+07
# HELP go_memstats_sys_bytes Number of bytes obtained from system.
# TYPE go_memstats_sys_bytes gauge
go_memstats_sys_bytes 1.9731492488e+10
# HELP go_threads Number of OS threads created.
# TYPE go_threads gauge
go_threads 84
# HELP golang_lru_cache_hit
# TYPE golang_lru_cache_hit counter
golang_lru_cache_hit{cache="attestations"} 1336
golang_lru_cache_hit{cache="beacon_active_validators_cache"} 1.198285e+06
golang_lru_cache_hit{cache="beacon_shuffled_sets_cache"} 405044
# HELP golang_lru_cache_miss
# TYPE golang_lru_cache_miss counter
golang_lru_cache_miss{cache="attestations"} 82
golang_lru_cache_miss{cache="beacon_active_validators_cache"} 2408
golang_lru_cache_miss{cache="beacon_shuffled_sets_cache"} 89
golang_lru_cache_miss{cache="validHashes"} 2646
# HELP libp2p_autonat_next_probe_timestamp Time of next probe
# TYPE libp2p_autonat_next_probe_timestamp gauge
libp2p_autonat_next_probe_timestamp 1.717157778e+09
# HELP libp2p_autonat_reachability_status Current node reachability
# TYPE libp2p_autonat_reachability_status gauge
libp2p_autonat_reachability_status 0
# HELP libp2p_autonat_reachability_status_confidence Node reachability status confidence
# TYPE libp2p_autonat_reachability_status_confidence gauge
libp2p_autonat_reachability_status_confidence 0
# HELP libp2p_autonat_received_dial_response_total Count of dial responses for client
# TYPE libp2p_autonat_received_dial_response_total counter
libp2p_autonat_received_dial_response_total{response_status="dial refused"} 996
# HELP libp2p_eventbus_events_emitted_total Events Emitted
# TYPE libp2p_eventbus_events_emitted_total counter
libp2p_eventbus_events_emitted_total{event="EvtLocalAddressesUpdated"} 1
libp2p_eventbus_events_emitted_total{event="EvtLocalProtocolsUpdated"} 18
libp2p_eventbus_events_emitted_total{event="EvtPeerConnectednessChanged"} 981125
libp2p_eventbus_events_emitted_total{event="EvtPeerIdentificationCompleted"} 170386
libp2p_eventbus_events_emitted_total{event="EvtPeerIdentificationFailed"} 320248
libp2p_eventbus_events_emitted_total{event="EvtPeerProtocolsUpdated"} 39866
# HELP libp2p_eventbus_subscriber_event_queued Event Queued for subscriber
# TYPE libp2p_eventbus_subscriber_event_queued counter
libp2p_eventbus_subscriber_event_queued{subscriber_name="autonat"} 170387
libp2p_eventbus_subscriber_event_queued{subscriber_name="identify (loop)"} 17
libp2p_eventbus_subscriber_event_queued{subscriber_name="pstoremanager"} 981125
# HELP libp2p_eventbus_subscriber_queue_full Subscriber Queue completely full
# TYPE libp2p_eventbus_subscriber_queue_full gauge
libp2p_eventbus_subscriber_queue_full{subscriber_name="autonat"} 0
libp2p_eventbus_subscriber_queue_full{subscriber_name="identify (loop)"} 0
libp2p_eventbus_subscriber_queue_full{subscriber_name="pstoremanager"} 0
# HELP libp2p_eventbus_subscriber_queue_length Subscriber queue length
# TYPE libp2p_eventbus_subscriber_queue_length gauge
libp2p_eventbus_subscriber_queue_length{subscriber_name="autonat"} 1
libp2p_eventbus_subscriber_queue_length{subscriber_name="identify (loop)"} 12
libp2p_eventbus_subscriber_queue_length{subscriber_name="pstoremanager"} 1
# HELP libp2p_eventbus_subscribers_total Number of subscribers for an event type
# TYPE libp2p_eventbus_subscribers_total gauge
libp2p_eventbus_subscribers_total{event="EvtLocalAddressesUpdated"} 2
libp2p_eventbus_subscribers_total{event="EvtLocalProtocolsUpdated"} 1
libp2p_eventbus_subscribers_total{event="EvtLocalReachabilityChanged"} 1
libp2p_eventbus_subscribers_total{event="EvtPeerConnectednessChanged"} 1
libp2p_eventbus_subscribers_total{event="EvtPeerIdentificationCompleted"} 1
# HELP libp2p_identify_addrs_count Address Count
# TYPE libp2p_identify_addrs_count gauge
libp2p_identify_addrs_count 0
# HELP libp2p_identify_addrs_received Number of addrs received
# TYPE libp2p_identify_addrs_received histogram
libp2p_identify_addrs_received_bucket{le="1"} 93612
libp2p_identify_addrs_received_bucket{le="2"} 132067
libp2p_identify_addrs_received_bucket{le="3"} 174550
libp2p_identify_addrs_received_bucket{le="4"} 193637
libp2p_identify_addrs_received_bucket{le="5"} 193760
libp2p_identify_addrs_received_bucket{le="6"} 205380
libp2p_identify_addrs_received_bucket{le="7"} 205386
libp2p_identify_addrs_received_bucket{le="8"} 208043
libp2p_identify_addrs_received_bucket{le="9"} 208043
libp2p_identify_addrs_received_bucket{le="10"} 209161
libp2p_identify_addrs_received_bucket{le="11"} 209161
libp2p_identify_addrs_received_bucket{le="12"} 209574
libp2p_identify_addrs_received_bucket{le="13"} 209574
libp2p_identify_addrs_received_bucket{le="14"} 209696
libp2p_identify_addrs_received_bucket{le="15"} 209696
libp2p_identify_addrs_received_bucket{le="16"} 209798
libp2p_identify_addrs_received_bucket{le="17"} 209798
libp2p_identify_addrs_received_bucket{le="18"} 209819
libp2p_identify_addrs_received_bucket{le="19"} 209819
libp2p_identify_addrs_received_bucket{le="20"} 209835
libp2p_identify_addrs_received_bucket{le="25"} 210137
libp2p_identify_addrs_received_bucket{le="30"} 210175
libp2p_identify_addrs_received_bucket{le="35"} 210175
libp2p_identify_addrs_received_bucket{le="40"} 210245
libp2p_identify_addrs_received_bucket{le="45"} 210245
libp2p_identify_addrs_received_bucket{le="50"} 210246
libp2p_identify_addrs_received_bucket{le="55"} 210246
libp2p_identify_addrs_received_bucket{le="60"} 210246
libp2p_identify_addrs_received_bucket{le="65"} 210250
libp2p_identify_addrs_received_bucket{le="70"} 210252
libp2p_identify_addrs_received_bucket{le="75"} 210252
libp2p_identify_addrs_received_bucket{le="80"} 210252
libp2p_identify_addrs_received_bucket{le="85"} 210252
libp2p_identify_addrs_received_bucket{le="90"} 210252
libp2p_identify_addrs_received_bucket{le="95"} 210252
libp2p_identify_addrs_received_bucket{le="100"} 210252
libp2p_identify_addrs_received_bucket{le="+Inf"} 210252
libp2p_identify_addrs_received_sum 497098
libp2p_identify_addrs_received_count 210252
# HELP libp2p_identify_conn_push_support_total Identify Connection Push Support
# TYPE libp2p_identify_conn_push_support_total counter
libp2p_identify_conn_push_support_total{support="not supported"} 83571
libp2p_identify_conn_push_support_total{support="supported"} 126154
# HELP libp2p_identify_identify_push_total Identify Push
# TYPE libp2p_identify_identify_push_total counter
libp2p_identify_identify_push_total{dir="inbound"} 39866
# HELP libp2p_identify_identify_pushes_triggered_total Pushes Triggered
# TYPE libp2p_identify_identify_pushes_triggered_total counter
libp2p_identify_identify_pushes_triggered_total{trigger="protocols_updated"} 4
# HELP libp2p_identify_identify_total Identify
# TYPE libp2p_identify_identify_total counter
libp2p_identify_identify_total{dir="inbound"} 100323
libp2p_identify_identify_total{dir="outbound"} 170386
# HELP libp2p_identify_protocols_count Protocols Count
# TYPE libp2p_identify_protocols_count gauge
libp2p_identify_protocols_count 18
# HELP libp2p_identify_protocols_received Number of Protocols received
# TYPE libp2p_identify_protocols_received histogram
libp2p_identify_protocols_received_bucket{le="1"} 57
libp2p_identify_protocols_received_bucket{le="2"} 57
libp2p_identify_protocols_received_bucket{le="3"} 79228
libp2p_identify_protocols_received_bucket{le="4"} 79228
libp2p_identify_protocols_received_bucket{le="5"} 79228
libp2p_identify_protocols_received_bucket{le="6"} 79229
libp2p_identify_protocols_received_bucket{le="7"} 80285
libp2p_identify_protocols_received_bucket{le="8"} 104570
libp2p_identify_protocols_received_bucket{le="9"} 107863
libp2p_identify_protocols_received_bucket{le="10"} 107863
libp2p_identify_protocols_received_bucket{le="11"} 117994
libp2p_identify_protocols_received_bucket{le="12"} 118005
libp2p_identify_protocols_received_bucket{le="13"} 171427
libp2p_identify_protocols_received_bucket{le="14"} 171510
libp2p_identify_protocols_received_bucket{le="15"} 207445
libp2p_identify_protocols_received_bucket{le="16"} 208924
libp2p_identify_protocols_received_bucket{le="17"} 208944
libp2p_identify_protocols_received_bucket{le="18"} 210252
libp2p_identify_protocols_received_bucket{le="19"} 210252
libp2p_identify_protocols_received_bucket{le="20"} 210252
libp2p_identify_protocols_received_bucket{le="25"} 210252
libp2p_identify_protocols_received_bucket{le="30"} 210252
libp2p_identify_protocols_received_bucket{le="35"} 210252
libp2p_identify_protocols_received_bucket{le="40"} 210252
libp2p_identify_protocols_received_bucket{le="45"} 210252
libp2p_identify_protocols_received_bucket{le="50"} 210252
libp2p_identify_protocols_received_bucket{le="55"} 210252
libp2p_identify_protocols_received_bucket{le="60"} 210252
libp2p_identify_protocols_received_bucket{le="65"} 210252
libp2p_identify_protocols_received_bucket{le="70"} 210252
libp2p_identify_protocols_received_bucket{le="75"} 210252
libp2p_identify_protocols_received_bucket{le="80"} 210252
libp2p_identify_protocols_received_bucket{le="85"} 210252
libp2p_identify_protocols_received_bucket{le="90"} 210252
libp2p_identify_protocols_received_bucket{le="95"} 210252
libp2p_identify_protocols_received_bucket{le="100"} 210252
libp2p_identify_protocols_received_bucket{le="+Inf"} 210252
libp2p_identify_protocols_received_sum 1.862622e+06
libp2p_identify_protocols_received_count 210252
# HELP libp2p_rcmgr_blocked_resources Number of blocked resources
# TYPE libp2p_rcmgr_blocked_resources gauge
libp2p_rcmgr_blocked_resources{dir="",resource="fd",scope="peer"} 997
libp2p_rcmgr_blocked_resources{dir="outbound",resource="connection",scope="peer"} 997
# HELP libp2p_rcmgr_conn_memory How many conns have reserved this bucket of memory, as reported to the Resource Manager
# TYPE libp2p_rcmgr_conn_memory histogram
libp2p_rcmgr_conn_memory_bucket{le="1024"} 0
libp2p_rcmgr_conn_memory_bucket{le="4096"} 0
libp2p_rcmgr_conn_memory_bucket{le="32768"} 0
libp2p_rcmgr_conn_memory_bucket{le="1.048576e+06"} 0
libp2p_rcmgr_conn_memory_bucket{le="3.3554432e+07"} 0
libp2p_rcmgr_conn_memory_bucket{le="2.68435456e+08"} 0
libp2p_rcmgr_conn_memory_bucket{le="5.36870912e+08"} 0
libp2p_rcmgr_conn_memory_bucket{le="1.073741824e+09"} 0
libp2p_rcmgr_conn_memory_bucket{le="2.147483648e+09"} 0
libp2p_rcmgr_conn_memory_bucket{le="4.294967296e+09"} 0
libp2p_rcmgr_conn_memory_bucket{le="+Inf"} 0
libp2p_rcmgr_conn_memory_sum 0
libp2p_rcmgr_conn_memory_count 0
# HELP libp2p_rcmgr_connections Number of Connections
# TYPE libp2p_rcmgr_connections gauge
libp2p_rcmgr_connections{dir="inbound",scope="system"} 0
libp2p_rcmgr_connections{dir="inbound",scope="transient"} 0
libp2p_rcmgr_connections{dir="outbound",scope="system"} 73
libp2p_rcmgr_connections{dir="outbound",scope="transient"} 0
# HELP libp2p_rcmgr_fds Number of file descriptors reserved as reported to the Resource Manager
# TYPE libp2p_rcmgr_fds gauge
libp2p_rcmgr_fds{scope="system"} 73
libp2p_rcmgr_fds{scope="transient"} 0
# HELP libp2p_rcmgr_memory Amount of memory reserved as reported to the Resource Manager
# TYPE libp2p_rcmgr_memory gauge
libp2p_rcmgr_memory{protocol="",scope="system"} 2.174976e+06
libp2p_rcmgr_memory{protocol="/ipfs/id/1.0.0",scope="protocol"} 0
libp2p_rcmgr_memory{protocol="/ipfs/id/push/1.0.0",scope="protocol"} 0
libp2p_rcmgr_memory{protocol="/libp2p/autonat/1.0.0",scope="protocol"} 0
# HELP libp2p_rcmgr_peer_connections Number of connections this peer has
# TYPE libp2p_rcmgr_peer_connections histogram
libp2p_rcmgr_peer_connections_bucket{dir="inbound",le="1"} 0
libp2p_rcmgr_peer_connections_bucket{dir="inbound",le="2"} 0
libp2p_rcmgr_peer_connections_bucket{dir="inbound",le="3"} 0
libp2p_rcmgr_peer_connections_bucket{dir="inbound",le="4"} 0
libp2p_rcmgr_peer_connections_bucket{dir="inbound",le="5"} 0
libp2p_rcmgr_peer_connections_bucket{dir="inbound",le="6"} 0
libp2p_rcmgr_peer_connections_bucket{dir="inbound",le="7"} 0
libp2p_rcmgr_peer_connections_bucket{dir="inbound",le="8"} 0
libp2p_rcmgr_peer_connections_bucket{dir="inbound",le="9"} 0
libp2p_rcmgr_peer_connections_bucket{dir="inbound",le="10"} 0
libp2p_rcmgr_peer_connections_bucket{dir="inbound",le="16"} 0
libp2p_rcmgr_peer_connections_bucket{dir="inbound",le="32"} 0
libp2p_rcmgr_peer_connections_bucket{dir="inbound",le="64"} 0
libp2p_rcmgr_peer_connections_bucket{dir="inbound",le="128"} 0
libp2p_rcmgr_peer_connections_bucket{dir="inbound",le="256"} 0
libp2p_rcmgr_peer_connections_bucket{dir="inbound",le="+Inf"} 0
libp2p_rcmgr_peer_connections_sum{dir="inbound"} 0
libp2p_rcmgr_peer_connections_count{dir="inbound"} 0
libp2p_rcmgr_peer_connections_bucket{dir="outbound",le="1"} 776283
libp2p_rcmgr_peer_connections_bucket{dir="outbound",le="2"} 795544
libp2p_rcmgr_peer_connections_bucket{dir="outbound",le="3"} 801931
libp2p_rcmgr_peer_connections_bucket{dir="outbound",le="4"} 807985
libp2p_rcmgr_peer_connections_bucket{dir="outbound",le="5"} 814117
libp2p_rcmgr_peer_connections_bucket{dir="outbound",le="6"} 820876
libp2p_rcmgr_peer_connections_bucket{dir="outbound",le="7"} 832689
libp2p_rcmgr_peer_connections_bucket{dir="outbound",le="8"} 840845
libp2p_rcmgr_peer_connections_bucket{dir="outbound",le="9"} 840845
libp2p_rcmgr_peer_connections_bucket{dir="outbound",le="10"} 840845
libp2p_rcmgr_peer_connections_bucket{dir="outbound",le="16"} 840845
libp2p_rcmgr_peer_connections_bucket{dir="outbound",le="32"} 840845
libp2p_rcmgr_peer_connections_bucket{dir="outbound",le="64"} 840845
libp2p_rcmgr_peer_connections_bucket{dir="outbound",le="128"} 840845
libp2p_rcmgr_peer_connections_bucket{dir="outbound",le="256"} 840845
libp2p_rcmgr_peer_connections_bucket{dir="outbound",le="+Inf"} 840845
libp2p_rcmgr_peer_connections_sum{dir="outbound"} 1.077335e+06
libp2p_rcmgr_peer_connections_count{dir="outbound"} 840845
# HELP libp2p_rcmgr_peer_memory How many peers have reserved this bucket of memory, as reported to the Resource Manager
# TYPE libp2p_rcmgr_peer_memory histogram
libp2p_rcmgr_peer_memory_bucket{le="1024"} 0
libp2p_rcmgr_peer_memory_bucket{le="4096"} 21894
libp2p_rcmgr_peer_memory_bucket{le="32768"} 1.484636e+06
libp2p_rcmgr_peer_memory_bucket{le="1.048576e+06"} 2.441781e+06
libp2p_rcmgr_peer_memory_bucket{le="3.3554432e+07"} 2.453068e+06
libp2p_rcmgr_peer_memory_bucket{le="2.68435456e+08"} 2.453068e+06
libp2p_rcmgr_peer_memory_bucket{le="5.36870912e+08"} 2.453068e+06
libp2p_rcmgr_peer_memory_bucket{le="1.073741824e+09"} 2.453068e+06
libp2p_rcmgr_peer_memory_bucket{le="2.147483648e+09"} 2.453068e+06
libp2p_rcmgr_peer_memory_bucket{le="4.294967296e+09"} 2.453068e+06
libp2p_rcmgr_peer_memory_bucket{le="+Inf"} 2.453068e+06
libp2p_rcmgr_peer_memory_sum 1.04675323904e+11
libp2p_rcmgr_peer_memory_count 2.453068e+06
# HELP libp2p_rcmgr_peer_streams Number of streams this peer has
# TYPE libp2p_rcmgr_peer_streams histogram
libp2p_rcmgr_peer_streams_bucket{dir="inbound",le="1"} 495494
libp2p_rcmgr_peer_streams_bucket{dir="inbound",le="2"} 862385
libp2p_rcmgr_peer_streams_bucket{dir="inbound",le="3"} 927092
libp2p_rcmgr_peer_streams_bucket{dir="inbound",le="4"} 936018
libp2p_rcmgr_peer_streams_bucket{dir="inbound",le="5"} 938430
libp2p_rcmgr_peer_streams_bucket{dir="inbound",le="6"} 939235
libp2p_rcmgr_peer_streams_bucket{dir="inbound",le="7"} 939580
libp2p_rcmgr_peer_streams_bucket{dir="inbound",le="8"} 939744
libp2p_rcmgr_peer_streams_bucket{dir="inbound",le="9"} 939841
libp2p_rcmgr_peer_streams_bucket{dir="inbound",le="10"} 939909
libp2p_rcmgr_peer_streams_bucket{dir="inbound",le="16"} 939996
libp2p_rcmgr_peer_streams_bucket{dir="inbound",le="32"} 939999
libp2p_rcmgr_peer_streams_bucket{dir="inbound",le="64"} 939999
libp2p_rcmgr_peer_streams_bucket{dir="inbound",le="128"} 939999
libp2p_rcmgr_peer_streams_bucket{dir="inbound",le="256"} 939999
libp2p_rcmgr_peer_streams_bucket{dir="inbound",le="+Inf"} 939999
libp2p_rcmgr_peer_streams_sum{dir="inbound"} 1.482355e+06
libp2p_rcmgr_peer_streams_count{dir="inbound"} 939999
libp2p_rcmgr_peer_streams_bucket{dir="outbound",le="1"} 808493
libp2p_rcmgr_peer_streams_bucket{dir="outbound",le="2"} 1.31021e+06
libp2p_rcmgr_peer_streams_bucket{dir="outbound",le="3"} 1.562138e+06
libp2p_rcmgr_peer_streams_bucket{dir="outbound",le="4"} 1.568316e+06
libp2p_rcmgr_peer_streams_bucket{dir="outbound",le="5"} 1.570383e+06
libp2p_rcmgr_peer_streams_bucket{dir="outbound",le="6"} 1.571268e+06
libp2p_rcmgr_peer_streams_bucket{dir="outbound",le="7"} 1.571653e+06
libp2p_rcmgr_peer_streams_bucket{dir="outbound",le="8"} 1.571807e+06
libp2p_rcmgr_peer_streams_bucket{dir="outbound",le="9"} 1.571865e+06
libp2p_rcmgr_peer_streams_bucket{dir="outbound",le="10"} 1.571891e+06
libp2p_rcmgr_peer_streams_bucket{dir="outbound",le="16"} 1.571908e+06
libp2p_rcmgr_peer_streams_bucket{dir="outbound",le="32"} 1.571908e+06
libp2p_rcmgr_peer_streams_bucket{dir="outbound",le="64"} 1.571908e+06
libp2p_rcmgr_peer_streams_bucket{dir="outbound",le="128"} 1.571908e+06
libp2p_rcmgr_peer_streams_bucket{dir="outbound",le="256"} 1.571908e+06
libp2p_rcmgr_peer_streams_bucket{dir="outbound",le="+Inf"} 1.571908e+06
libp2p_rcmgr_peer_streams_sum{dir="outbound"} 2.612973e+06
libp2p_rcmgr_peer_streams_count{dir="outbound"} 1.571908e+06
# HELP libp2p_rcmgr_previous_conn_memory How many conns have previously reserved this bucket of memory, as reported to the Resource Manager
# TYPE libp2p_rcmgr_previous_conn_memory histogram
libp2p_rcmgr_previous_conn_memory_bucket{le="1024"} 0
libp2p_rcmgr_previous_conn_memory_bucket{le="4096"} 0
libp2p_rcmgr_previous_conn_memory_bucket{le="32768"} 0
libp2p_rcmgr_previous_conn_memory_bucket{le="1.048576e+06"} 0
libp2p_rcmgr_previous_conn_memory_bucket{le="3.3554432e+07"} 0
libp2p_rcmgr_previous_conn_memory_bucket{le="2.68435456e+08"} 0
libp2p_rcmgr_previous_conn_memory_bucket{le="5.36870912e+08"} 0
libp2p_rcmgr_previous_conn_memory_bucket{le="1.073741824e+09"} 0
libp2p_rcmgr_previous_conn_memory_bucket{le="2.147483648e+09"} 0
libp2p_rcmgr_previous_conn_memory_bucket{le="4.294967296e+09"} 0
libp2p_rcmgr_previous_conn_memory_bucket{le="+Inf"} 0
libp2p_rcmgr_previous_conn_memory_sum 0
libp2p_rcmgr_previous_conn_memory_count 0
# HELP libp2p_rcmgr_previous_peer_connections Number of connections this peer previously had. This is used to get the current connection number per peer histogram by subtracting this from the peer_connections histogram
# TYPE libp2p_rcmgr_previous_peer_connections histogram
libp2p_rcmgr_previous_peer_connections_bucket{dir="inbound",le="1"} 0
libp2p_rcmgr_previous_peer_connections_bucket{dir="inbound",le="2"} 0
libp2p_rcmgr_previous_peer_connections_bucket{dir="inbound",le="3"} 0
libp2p_rcmgr_previous_peer_connections_bucket{dir="inbound",le="4"} 0
libp2p_rcmgr_previous_peer_connections_bucket{dir="inbound",le="5"} 0
libp2p_rcmgr_previous_peer_connections_bucket{dir="inbound",le="6"} 0
libp2p_rcmgr_previous_peer_connections_bucket{dir="inbound",le="7"} 0
libp2p_rcmgr_previous_peer_connections_bucket{dir="inbound",le="8"} 0
libp2p_rcmgr_previous_peer_connections_bucket{dir="inbound",le="9"} 0
libp2p_rcmgr_previous_peer_connections_bucket{dir="inbound",le="10"} 0
libp2p_rcmgr_previous_peer_connections_bucket{dir="inbound",le="16"} 0
libp2p_rcmgr_previous_peer_connections_bucket{dir="inbound",le="32"} 0
libp2p_rcmgr_previous_peer_connections_bucket{dir="inbound",le="64"} 0
libp2p_rcmgr_previous_peer_connections_bucket{dir="inbound",le="128"} 0
libp2p_rcmgr_previous_peer_connections_bucket{dir="inbound",le="256"} 0
libp2p_rcmgr_previous_peer_connections_bucket{dir="inbound",le="+Inf"} 0
libp2p_rcmgr_previous_peer_connections_sum{dir="inbound"} 0
libp2p_rcmgr_previous_peer_connections_count{dir="inbound"} 0
libp2p_rcmgr_previous_peer_connections_bucket{dir="outbound",le="1"} 776210
libp2p_rcmgr_previous_peer_connections_bucket{dir="outbound",le="2"} 795471
libp2p_rcmgr_previous_peer_connections_bucket{dir="outbound",le="3"} 801858
libp2p_rcmgr_previous_peer_connections_bucket{dir="outbound",le="4"} 807912
libp2p_rcmgr_previous_peer_connections_bucket{dir="outbound",le="5"} 814044
libp2p_rcmgr_previous_peer_connections_bucket{dir="outbound",le="6"} 820803
libp2p_rcmgr_previous_peer_connections_bucket{dir="outbound",le="7"} 832616
libp2p_rcmgr_previous_peer_connections_bucket{dir="outbound",le="8"} 840772
libp2p_rcmgr_previous_peer_connections_bucket{dir="outbound",le="9"} 840772
libp2p_rcmgr_previous_peer_connections_bucket{dir="outbound",le="10"} 840772
libp2p_rcmgr_previous_peer_connections_bucket{dir="outbound",le="16"} 840772
libp2p_rcmgr_previous_peer_connections_bucket{dir="outbound",le="32"} 840772
libp2p_rcmgr_previous_peer_connections_bucket{dir="outbound",le="64"} 840772
libp2p_rcmgr_previous_peer_connections_bucket{dir="outbound",le="128"} 840772
libp2p_rcmgr_previous_peer_connections_bucket{dir="outbound",le="256"} 840772
libp2p_rcmgr_previous_peer_connections_bucket{dir="outbound",le="+Inf"} 840772
libp2p_rcmgr_previous_peer_connections_sum{dir="outbound"} 1.077262e+06
libp2p_rcmgr_previous_peer_connections_count{dir="outbound"} 840772
# HELP libp2p_rcmgr_previous_peer_memory How many peers have previously reserved this bucket of memory, as reported to the Resource Manager
# TYPE libp2p_rcmgr_previous_peer_memory histogram
libp2p_rcmgr_previous_peer_memory_bucket{le="1024"} 0
libp2p_rcmgr_previous_peer_memory_bucket{le="4096"} 21894
libp2p_rcmgr_previous_peer_memory_bucket{le="32768"} 1.484636e+06
libp2p_rcmgr_previous_peer_memory_bucket{le="1.048576e+06"} 2.441722e+06
libp2p_rcmgr_previous_peer_memory_bucket{le="3.3554432e+07"} 2.453009e+06
libp2p_rcmgr_previous_peer_memory_bucket{le="2.68435456e+08"} 2.453009e+06
libp2p_rcmgr_previous_peer_memory_bucket{le="5.36870912e+08"} 2.453009e+06
libp2p_rcmgr_previous_peer_memory_bucket{le="1.073741824e+09"} 2.453009e+06
libp2p_rcmgr_previous_peer_memory_bucket{le="2.147483648e+09"} 2.453009e+06
libp2p_rcmgr_previous_peer_memory_bucket{le="4.294967296e+09"} 2.453009e+06
libp2p_rcmgr_previous_peer_memory_bucket{le="+Inf"} 2.453009e+06
libp2p_rcmgr_previous_peer_memory_sum 1.04673148928e+11
libp2p_rcmgr_previous_peer_memory_count 2.453009e+06
# HELP libp2p_rcmgr_previous_peer_streams Number of streams this peer has
# TYPE libp2p_rcmgr_previous_peer_streams histogram
libp2p_rcmgr_previous_peer_streams_bucket{dir="inbound",le="1"} 495435
libp2p_rcmgr_previous_peer_streams_bucket{dir="inbound",le="2"} 862326
libp2p_rcmgr_previous_peer_streams_bucket{dir="inbound",le="3"} 927033
libp2p_rcmgr_previous_peer_streams_bucket{dir="inbound",le="4"} 935959
libp2p_rcmgr_previous_peer_streams_bucket{dir="inbound",le="5"} 938371
libp2p_rcmgr_previous_peer_streams_bucket{dir="inbound",le="6"} 939176
libp2p_rcmgr_previous_peer_streams_bucket{dir="inbound",le="7"} 939521
libp2p_rcmgr_previous_peer_streams_bucket{dir="inbound",le="8"} 939685
libp2p_rcmgr_previous_peer_streams_bucket{dir="inbound",le="9"} 939782
libp2p_rcmgr_previous_peer_streams_bucket{dir="inbound",le="10"} 939850
libp2p_rcmgr_previous_peer_streams_bucket{dir="inbound",le="16"} 939937
libp2p_rcmgr_previous_peer_streams_bucket{dir="inbound",le="32"} 939940
libp2p_rcmgr_previous_peer_streams_bucket{dir="inbound",le="64"} 939940
libp2p_rcmgr_previous_peer_streams_bucket{dir="inbound",le="128"} 939940
libp2p_rcmgr_previous_peer_streams_bucket{dir="inbound",le="256"} 939940
libp2p_rcmgr_previous_peer_streams_bucket{dir="inbound",le="+Inf"} 939940
libp2p_rcmgr_previous_peer_streams_sum{dir="inbound"} 1.482296e+06
libp2p_rcmgr_previous_peer_streams_count{dir="inbound"} 939940
libp2p_rcmgr_previous_peer_streams_bucket{dir="outbound",le="1"} 808435
libp2p_rcmgr_previous_peer_streams_bucket{dir="outbound",le="2"} 1.310151e+06
libp2p_rcmgr_previous_peer_streams_bucket{dir="outbound",le="3"} 1.562079e+06
libp2p_rcmgr_previous_peer_streams_bucket{dir="outbound",le="4"} 1.568257e+06
libp2p_rcmgr_previous_peer_streams_bucket{dir="outbound",le="5"} 1.570324e+06
libp2p_rcmgr_previous_peer_streams_bucket{dir="outbound",le="6"} 1.571209e+06
libp2p_rcmgr_previous_peer_streams_bucket{dir="outbound",le="7"} 1.571594e+06
libp2p_rcmgr_previous_peer_streams_bucket{dir="outbound",le="8"} 1.571748e+06
libp2p_rcmgr_previous_peer_streams_bucket{dir="outbound",le="9"} 1.571806e+06
libp2p_rcmgr_previous_peer_streams_bucket{dir="outbound",le="10"} 1.571832e+06
libp2p_rcmgr_previous_peer_streams_bucket{dir="outbound",le="16"} 1.571849e+06
libp2p_rcmgr_previous_peer_streams_bucket{dir="outbound",le="32"} 1.571849e+06
libp2p_rcmgr_previous_peer_streams_bucket{dir="outbound",le="64"} 1.571849e+06
libp2p_rcmgr_previous_peer_streams_bucket{dir="outbound",le="128"} 1.571849e+06
libp2p_rcmgr_previous_peer_streams_bucket{dir="outbound",le="256"} 1.571849e+06
libp2p_rcmgr_previous_peer_streams_bucket{dir="outbound",le="+Inf"} 1.571849e+06
libp2p_rcmgr_previous_peer_streams_sum{dir="outbound"} 2.612913e+06
libp2p_rcmgr_previous_peer_streams_count{dir="outbound"} 1.571849e+06
# HELP libp2p_rcmgr_streams Number of Streams
# TYPE libp2p_rcmgr_streams gauge
libp2p_rcmgr_streams{dir="inbound",protocol="",scope="system"} 59
libp2p_rcmgr_streams{dir="inbound",protocol="",scope="transient"} 0
libp2p_rcmgr_streams{dir="inbound",protocol="/eth2/beacon_chain/req/beacon_blocks_by_range/2/ssz_snappy",scope="protocol"} 0
libp2p_rcmgr_streams{dir="inbound",protocol="/eth2/beacon_chain/req/beacon_blocks_by_root/2/ssz_snappy",scope="protocol"} 0
libp2p_rcmgr_streams{dir="inbound",protocol="/eth2/beacon_chain/req/blob_sidecars_by_range/1/ssz_snappy",scope="protocol"} 0
libp2p_rcmgr_streams{dir="inbound",protocol="/eth2/beacon_chain/req/blob_sidecars_by_root/1/ssz_snappy",scope="protocol"} 0
libp2p_rcmgr_streams{dir="inbound",protocol="/eth2/beacon_chain/req/goodbye/1/ssz_snappy",scope="protocol"} 0
libp2p_rcmgr_streams{dir="inbound",protocol="/eth2/beacon_chain/req/metadata/2/ssz_snappy",scope="protocol"} 0
libp2p_rcmgr_streams{dir="inbound",protocol="/eth2/beacon_chain/req/ping/1/ssz_snappy",scope="protocol"} 0
libp2p_rcmgr_streams{dir="inbound",protocol="/eth2/beacon_chain/req/status/1/ssz_snappy",scope="protocol"} 0
libp2p_rcmgr_streams{dir="inbound",protocol="/ipfs/id/1.0.0",scope="protocol"} 0
libp2p_rcmgr_streams{dir="inbound",protocol="/ipfs/id/push/1.0.0",scope="protocol"} 0
libp2p_rcmgr_streams{dir="inbound",protocol="/meshsub/1.1.0",scope="protocol"} 59
libp2p_rcmgr_streams{dir="outbound",protocol="",scope="system"} 60
libp2p_rcmgr_streams{dir="outbound",protocol="",scope="transient"} 0
libp2p_rcmgr_streams{dir="outbound",protocol="/eth2/beacon_chain/req/beacon_blocks_by_range/2/ssz_snappy",scope="protocol"} 0
libp2p_rcmgr_streams{dir="outbound",protocol="/eth2/beacon_chain/req/blob_sidecars_by_root/1/ssz_snappy",scope="protocol"} 1
libp2p_rcmgr_streams{dir="outbound",protocol="/eth2/beacon_chain/req/status/1/ssz_snappy",scope="protocol"} 0
libp2p_rcmgr_streams{dir="outbound",protocol="/ipfs/id/1.0.0",scope="protocol"} 0
libp2p_rcmgr_streams{dir="outbound",protocol="/libp2p/autonat/1.0.0",scope="protocol"} 0
libp2p_rcmgr_streams{dir="outbound",protocol="/meshsub/1.1.0",scope="protocol"} 59
# HELP libp2p_swarm_black_hole_filter_next_request_allowed_after Number of requests after which the next request will be allowed
# TYPE libp2p_swarm_black_hole_filter_next_request_allowed_after gauge
libp2p_swarm_black_hole_filter_next_request_allowed_after{name="IPv6"} 11
# HELP libp2p_swarm_black_hole_filter_state State of the black hole filter
# TYPE libp2p_swarm_black_hole_filter_state gauge
libp2p_swarm_black_hole_filter_state{name="IPv6"} 2
# HELP libp2p_swarm_black_hole_filter_success_fraction Fraction of successful dials among the last n requests
# TYPE libp2p_swarm_black_hole_filter_success_fraction gauge
libp2p_swarm_black_hole_filter_success_fraction{name="IPv6"} 0
# HELP libp2p_swarm_connection_duration_seconds Duration of a Connection
# TYPE libp2p_swarm_connection_duration_seconds histogram
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.0625"} 486
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.125"} 45631
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.25"} 184565
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.5"} 225985
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="1"} 367257
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="2"} 397194
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="4"} 402154
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="8"} 404423
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="16"} 407471
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="32"} 427581
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="64"} 427947
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="128"} 428124
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="256"} 428215
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="512"} 428260
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="1024"} 428323
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="2048"} 428362
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="4096"} 428368
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="8192"} 428374
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="16384"} 428375
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="32768"} 428375
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="65536"} 428375
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="131072"} 428375
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="262144"} 428375
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="524288"} 428375
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="1.048576e+06"} 428375
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="+Inf"} 428375
libp2p_swarm_connection_duration_seconds_sum{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp"} 988107.4760285843
libp2p_swarm_connection_duration_seconds_count{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp"} 428375
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.0625"} 54
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.125"} 4600
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.25"} 27144
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.5"} 36789
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="1"} 46739
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="2"} 49834
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="4"} 50203
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="8"} 50558
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="16"} 50684
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="32"} 50828
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="64"} 51531
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="128"} 53357
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="256"} 57601
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="512"} 58038
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="1024"} 58095
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="2048"} 58099
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="4096"} 58101
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="8192"} 58102
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="16384"} 58102
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="32768"} 58102
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="65536"} 58102
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="131072"} 58102
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="262144"} 58102
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="524288"} 58102
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="1.048576e+06"} 58102
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="+Inf"} 58102
libp2p_swarm_connection_duration_seconds_sum{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp"} 1.1843596195995877e+06
libp2p_swarm_connection_duration_seconds_count{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp"} 58102
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="0.0625"} 0
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="0.125"} 451
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="0.25"} 1899
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="0.5"} 3149
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="1"} 3820
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="2"} 4094
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="4"} 4119
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="8"} 4119
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="16"} 4119
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="32"} 4119
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="64"} 4119
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="128"} 4119
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="256"} 4119
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="512"} 4119
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="1024"} 4119
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="2048"} 4119
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="4096"} 4119
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="8192"} 4119
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="16384"} 4119
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="32768"} 4119
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="65536"} 4119
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="131072"} 4119
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="262144"} 4119
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="524288"} 4119
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="1.048576e+06"} 4119
libp2p_swarm_connection_duration_seconds_bucket{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="+Inf"} 4119
libp2p_swarm_connection_duration_seconds_sum{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp"} 1672.8701925719995
libp2p_swarm_connection_duration_seconds_count{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp"} 4119
# HELP libp2p_swarm_connections_closed_total Connections Closed
# TYPE libp2p_swarm_connections_closed_total counter
libp2p_swarm_connections_closed_total{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp"} 428375
libp2p_swarm_connections_closed_total{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp"} 58102
libp2p_swarm_connections_closed_total{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp"} 4119
# HELP libp2p_swarm_connections_opened_total Connections Opened
# TYPE libp2p_swarm_connections_opened_total counter
libp2p_swarm_connections_opened_total{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp"} 428399
libp2p_swarm_connections_opened_total{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp"} 58137
libp2p_swarm_connections_opened_total{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp"} 4119
# HELP libp2p_swarm_dial_errors_total Dial Error
# TYPE libp2p_swarm_dial_errors_total counter
libp2p_swarm_dial_errors_total{error="canceled: concurrent dial successful",ip_version="ip4",transport="tcp"} 23899
libp2p_swarm_dial_errors_total{error="connection refused",ip_version="ip4",transport="tcp"} 19258
libp2p_swarm_dial_errors_total{error="connection refused",ip_version="ip6",transport="tcp"} 52
libp2p_swarm_dial_errors_total{error="deadline",ip_version="ip4",transport="tcp"} 183069
libp2p_swarm_dial_errors_total{error="deadline",ip_version="ip6",transport="tcp"} 1
libp2p_swarm_dial_errors_total{error="other",ip_version="ip4",transport="tcp"} 84051
libp2p_swarm_dial_errors_total{error="other",ip_version="ip6",transport="tcp"} 53
libp2p_swarm_dial_errors_total{error="timeout",ip_version="ip4",transport="tcp"} 560
# HELP libp2p_swarm_dial_ranking_delay_seconds delay introduced by the dial ranking logic
# TYPE libp2p_swarm_dial_ranking_delay_seconds histogram
libp2p_swarm_dial_ranking_delay_seconds_bucket{le="0.001"} 490405
libp2p_swarm_dial_ranking_delay_seconds_bucket{le="0.01"} 490565
libp2p_swarm_dial_ranking_delay_seconds_bucket{le="0.05"} 490567
libp2p_swarm_dial_ranking_delay_seconds_bucket{le="0.1"} 490567
libp2p_swarm_dial_ranking_delay_seconds_bucket{le="0.2"} 490568
libp2p_swarm_dial_ranking_delay_seconds_bucket{le="0.3"} 490568
libp2p_swarm_dial_ranking_delay_seconds_bucket{le="0.4"} 490568
libp2p_swarm_dial_ranking_delay_seconds_bucket{le="0.5"} 490568
libp2p_swarm_dial_ranking_delay_seconds_bucket{le="0.75"} 490569
libp2p_swarm_dial_ranking_delay_seconds_bucket{le="1"} 490571
libp2p_swarm_dial_ranking_delay_seconds_bucket{le="2"} 490574
libp2p_swarm_dial_ranking_delay_seconds_bucket{le="+Inf"} 490575
libp2p_swarm_dial_ranking_delay_seconds_sum 27.819637605999773
libp2p_swarm_dial_ranking_delay_seconds_count 490575
# HELP libp2p_swarm_dials_per_peer_total Number of addresses dialed per peer
# TYPE libp2p_swarm_dials_per_peer_total counter
libp2p_swarm_dials_per_peer_total{num_dials="0",outcome="failed"} 32322
libp2p_swarm_dials_per_peer_total{num_dials="1",outcome="failed"} 268727
libp2p_swarm_dials_per_peer_total{num_dials="1",outcome="success"} 477416
libp2p_swarm_dials_per_peer_total{num_dials="2",outcome="failed"} 1700
libp2p_swarm_dials_per_peer_total{num_dials="2",outcome="success"} 10004
libp2p_swarm_dials_per_peer_total{num_dials="3",outcome="failed"} 118
libp2p_swarm_dials_per_peer_total{num_dials="3",outcome="success"} 213
libp2p_swarm_dials_per_peer_total{num_dials="4",outcome="success"} 9
libp2p_swarm_dials_per_peer_total{num_dials="5",outcome="success"} 5
libp2p_swarm_dials_per_peer_total{num_dials=">=6",outcome="failed"} 51
libp2p_swarm_dials_per_peer_total{num_dials=">=6",outcome="success"} 2928
# HELP libp2p_swarm_handshake_latency_seconds Duration of the libp2p Handshake
# TYPE libp2p_swarm_handshake_latency_seconds histogram
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.001"} 0
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.0013000000000000002"} 0
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.0016900000000000003"} 0
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.0021970000000000006"} 1
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.0028561000000000007"} 1
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.003712930000000001"} 1
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.004826809000000001"} 1
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.006274851700000002"} 1
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.008157307210000003"} 1
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.010604499373000004"} 31
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.013785849184900006"} 112
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.01792160394037001"} 176
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.02329808512248101"} 204
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.030287510659225317"} 290
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.03937376385699291"} 337
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.05118589301409079"} 439
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.06654166091831802"} 763
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.08650415919381343"} 46591
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.11245540695195747"} 48062
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.14619202903754472"} 49461
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.19004963774880815"} 156910
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.2470645290734506"} 216605
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.3211838877954858"} 229949
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.41753905413413156"} 233302
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.542800770374371"} 269757
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.7056410014866824"} 370748
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.9173333019326871"} 402808
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="1.1925332925124934"} 415289
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="1.5502932802662415"} 424795
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="2.015381264346114"} 428022
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="2.6199956436499483"} 428238
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="3.405994336744933"} 428299
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="4.427792637768413"} 428341
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="5.7561304290989375"} 428363
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="7.482969557828619"} 428372
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="+Inf"} 428399
libp2p_swarm_handshake_latency_seconds_sum{early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp"} 174285.9950712504
libp2p_swarm_handshake_latency_seconds_count{early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp"} 428399
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.001"} 0
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.0013000000000000002"} 0
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.0016900000000000003"} 0
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.0021970000000000006"} 0
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.0028561000000000007"} 0
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.003712930000000001"} 0
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.004826809000000001"} 0
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.006274851700000002"} 0
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.008157307210000003"} 0
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.010604499373000004"} 11
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.013785849184900006"} 44
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.01792160394037001"} 90
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.02329808512248101"} 98
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.030287510659225317"} 104
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.03937376385699291"} 188
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.05118589301409079"} 280
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.06654166091831802"} 4904
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.08650415919381343"} 5268
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.11245540695195747"} 5378
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.14619202903754472"} 28710
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.19004963774880815"} 39540
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.2470645290734506"} 40785
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.3211838877954858"} 41076
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.41753905413413156"} 47625
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.542800770374371"} 51306
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.7056410014866824"} 54060
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="0.9173333019326871"} 56713
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="1.1925332925124934"} 57811
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="1.5502932802662415"} 58013
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="2.015381264346114"} 58115
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="2.6199956436499483"} 58125
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="3.405994336744933"} 58127
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="4.427792637768413"} 58133
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="5.7561304290989375"} 58136
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="7.482969557828619"} 58137
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp",le="+Inf"} 58137
libp2p_swarm_handshake_latency_seconds_sum{early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp"} 15289.445331019955
libp2p_swarm_handshake_latency_seconds_count{early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp"} 58137
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="0.001"} 0
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="0.0013000000000000002"} 0
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="0.0016900000000000003"} 0
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="0.0021970000000000006"} 0
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="0.0028561000000000007"} 0
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="0.003712930000000001"} 0
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="0.004826809000000001"} 0
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="0.006274851700000002"} 0
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="0.008157307210000003"} 0
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="0.010604499373000004"} 0
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="0.013785849184900006"} 0
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="0.01792160394037001"} 0
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="0.02329808512248101"} 0
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="0.030287510659225317"} 0
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="0.03937376385699291"} 0
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="0.05118589301409079"} 6
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="0.06654166091831802"} 453
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="0.08650415919381343"} 454
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="0.11245540695195747"} 464
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="0.14619202903754472"} 2402
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="0.19004963774880815"} 3099
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="0.2470645290734506"} 3146
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="0.3211838877954858"} 3161
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="0.41753905413413156"} 3482
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="0.542800770374371"} 3720
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="0.7056410014866824"} 3911
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="0.9173333019326871"} 4058
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="1.1925332925124934"} 4110
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="1.5502932802662415"} 4118
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="2.015381264346114"} 4119
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="2.6199956436499483"} 4119
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="3.405994336744933"} 4119
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="4.427792637768413"} 4119
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="5.7561304290989375"} 4119
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="7.482969557828619"} 4119
libp2p_swarm_handshake_latency_seconds_bucket{early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp",le="+Inf"} 4119
libp2p_swarm_handshake_latency_seconds_sum{early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp"} 939.3618615569995
libp2p_swarm_handshake_latency_seconds_count{early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp"} 4119
# HELP libp2p_swarm_key_types_total key type
# TYPE libp2p_swarm_key_types_total counter
libp2p_swarm_key_types_total{dir="outbound",key_type="Secp256k1"} 490655
# HELP mem_anonymous
# TYPE mem_anonymous gauge
mem_anonymous 1.9432056e+07
# HELP mem_private
# TYPE mem_private gauge
mem_private{type="clean"} 9.00518e+06
mem_private{type="dirty"} 1.9438196e+07
# HELP mem_pss
# TYPE mem_pss gauge
mem_pss 2.8443424e+07
# HELP mem_referenced
# TYPE mem_referenced gauge
mem_referenced 2.7070496e+07
# HELP mem_rss
# TYPE mem_rss gauge
mem_rss 2.8446108e+07
# HELP mem_shared
# TYPE mem_shared gauge
mem_shared{type="clean"} 2732
mem_shared{type="dirty"} 0
# HELP mem_size
# TYPE mem_size gauge
mem_size 0
# HELP mem_swap
# TYPE mem_swap gauge
mem_swap 106312
# HELP p2p_dials
# TYPE p2p_dials counter
p2p_dials 242030
# HELP p2p_egress
# TYPE p2p_egress counter
p2p_egress 2.472919517e+09
# HELP p2p_ingress
# TYPE p2p_ingress counter
p2p_ingress 4.982803472e+09
# HELP p2p_ingress_eth_66_0x00
# TYPE p2p_ingress_eth_66_0x00 gauge
p2p_ingress_eth_66_0x00 103
# HELP p2p_ingress_eth_66_0x00_packets
# TYPE p2p_ingress_eth_66_0x00_packets gauge
p2p_ingress_eth_66_0x00_packets 1
# HELP p2p_ingress_eth_66_0x03
# TYPE p2p_ingress_eth_66_0x03 gauge
p2p_ingress_eth_66_0x03 49
# HELP p2p_ingress_eth_66_0x03_packets
# TYPE p2p_ingress_eth_66_0x03_packets gauge
p2p_ingress_eth_66_0x03_packets 1
# HELP p2p_ingress_eth_66_0x04
# TYPE p2p_ingress_eth_66_0x04 gauge
p2p_ingress_eth_66_0x04 13
# HELP p2p_ingress_eth_66_0x04_packets
# TYPE p2p_ingress_eth_66_0x04_packets gauge
p2p_ingress_eth_66_0x04_packets 1
# HELP p2p_ingress_eth_66_0x09
# TYPE p2p_ingress_eth_66_0x09 gauge
p2p_ingress_eth_66_0x09 46
# HELP p2p_ingress_eth_66_0x09_packets
# TYPE p2p_ingress_eth_66_0x09_packets gauge
p2p_ingress_eth_66_0x09_packets 1
# HELP p2p_ingress_eth_67_0x00
# TYPE p2p_ingress_eth_67_0x00 gauge
p2p_ingress_eth_67_0x00 93
# HELP p2p_ingress_eth_67_0x00_packets
# TYPE p2p_ingress_eth_67_0x00_packets gauge
p2p_ingress_eth_67_0x00_packets 1
# HELP p2p_ingress_eth_67_0x02
# TYPE p2p_ingress_eth_67_0x02 gauge
p2p_ingress_eth_67_0x02 761
# HELP p2p_ingress_eth_67_0x02_packets
# TYPE p2p_ingress_eth_67_0x02_packets gauge
p2p_ingress_eth_67_0x02_packets 1
# HELP p2p_ingress_eth_67_0x03
# TYPE p2p_ingress_eth_67_0x03 gauge
p2p_ingress_eth_67_0x03 49
# HELP p2p_ingress_eth_67_0x03_packets
# TYPE p2p_ingress_eth_67_0x03_packets gauge
p2p_ingress_eth_67_0x03_packets 1
# HELP p2p_ingress_eth_67_0x04
# TYPE p2p_ingress_eth_67_0x04 gauge
p2p_ingress_eth_67_0x04 632
# HELP p2p_ingress_eth_67_0x04_packets
# TYPE p2p_ingress_eth_67_0x04_packets gauge
p2p_ingress_eth_67_0x04_packets 1
# HELP p2p_ingress_eth_67_0x08
# TYPE p2p_ingress_eth_67_0x08 gauge
p2p_ingress_eth_67_0x08 3341
# HELP p2p_ingress_eth_67_0x08_packets
# TYPE p2p_ingress_eth_67_0x08_packets gauge
p2p_ingress_eth_67_0x08_packets 1
# HELP p2p_ingress_eth_67_0x09
# TYPE p2p_ingress_eth_67_0x09 gauge
p2p_ingress_eth_67_0x09 38
# HELP p2p_ingress_eth_67_0x09_packets
# TYPE p2p_ingress_eth_67_0x09_packets gauge
p2p_ingress_eth_67_0x09_packets 1
# HELP p2p_ingress_eth_67_0x0a
# TYPE p2p_ingress_eth_67_0x0a gauge
p2p_ingress_eth_67_0x0a 910
# HELP p2p_ingress_eth_67_0x0a_packets
# TYPE p2p_ingress_eth_67_0x0a_packets gauge
p2p_ingress_eth_67_0x0a_packets 1
# HELP p2p_ingress_eth_68_0x00
# TYPE p2p_ingress_eth_68_0x00 gauge
p2p_ingress_eth_68_0x00 95
# HELP p2p_ingress_eth_68_0x00_packets
# TYPE p2p_ingress_eth_68_0x00_packets gauge
p2p_ingress_eth_68_0x00_packets 1
# HELP p2p_ingress_eth_68_0x02
# TYPE p2p_ingress_eth_68_0x02 gauge
p2p_ingress_eth_68_0x02 147
# HELP p2p_ingress_eth_68_0x02_packets
# TYPE p2p_ingress_eth_68_0x02_packets gauge
p2p_ingress_eth_68_0x02_packets 1
# HELP p2p_ingress_eth_68_0x03
# TYPE p2p_ingress_eth_68_0x03 gauge
p2p_ingress_eth_68_0x03 23
# HELP p2p_ingress_eth_68_0x03_packets
# TYPE p2p_ingress_eth_68_0x03_packets gauge
p2p_ingress_eth_68_0x03_packets 1
# HELP p2p_ingress_eth_68_0x04
# TYPE p2p_ingress_eth_68_0x04 gauge
p2p_ingress_eth_68_0x04 594
# HELP p2p_ingress_eth_68_0x04_packets
# TYPE p2p_ingress_eth_68_0x04_packets gauge
p2p_ingress_eth_68_0x04_packets 1
# HELP p2p_ingress_eth_68_0x05
# TYPE p2p_ingress_eth_68_0x05 gauge
p2p_ingress_eth_68_0x05 46
# HELP p2p_ingress_eth_68_0x05_packets
# TYPE p2p_ingress_eth_68_0x05_packets gauge
p2p_ingress_eth_68_0x05_packets 1
# HELP p2p_ingress_eth_68_0x08
# TYPE p2p_ingress_eth_68_0x08 gauge
p2p_ingress_eth_68_0x08 42
# HELP p2p_ingress_eth_68_0x08_packets
# TYPE p2p_ingress_eth_68_0x08_packets gauge
p2p_ingress_eth_68_0x08_packets 1
# HELP p2p_ingress_eth_68_0x09
# TYPE p2p_ingress_eth_68_0x09 gauge
p2p_ingress_eth_68_0x09 40
# HELP p2p_ingress_eth_68_0x09_packets
# TYPE p2p_ingress_eth_68_0x09_packets gauge
p2p_ingress_eth_68_0x09_packets 1
# HELP p2p_ingress_eth_68_0x0a
# TYPE p2p_ingress_eth_68_0x0a gauge
p2p_ingress_eth_68_0x0a 684
# HELP p2p_ingress_eth_68_0x0a_packets
# TYPE p2p_ingress_eth_68_0x0a_packets gauge
p2p_ingress_eth_68_0x0a_packets 1
# HELP p2p_ingress_eth_68_0x0f
# TYPE p2p_ingress_eth_68_0x0f gauge
p2p_ingress_eth_68_0x0f 4244
# HELP p2p_ingress_eth_68_0x0f_packets
# TYPE p2p_ingress_eth_68_0x0f_packets gauge
p2p_ingress_eth_68_0x0f_packets 1
# HELP p2p_peers
# TYPE p2p_peers gauge
p2p_peers 43
# HELP p2p_serves
# TYPE p2p_serves counter
p2p_serves 0
# HELP pool_add_remote_txs
# TYPE pool_add_remote_txs summary
pool_add_remote_txs{quantile="0.5"} 2.141e-06
pool_add_remote_txs{quantile="0.9"} 4.043e-06
pool_add_remote_txs{quantile="0.97"} 1.228e-05
pool_add_remote_txs{quantile="0.99"} 1.9788e-05
pool_add_remote_txs_sum 3.5964411890000516
pool_add_remote_txs_count 437348
# HELP pool_new_block
# TYPE pool_new_block summary
pool_new_block{quantile="0.5"} 0.004494071
pool_new_block{quantile="0.9"} 0.011528259
pool_new_block{quantile="0.97"} 0.041028634
pool_new_block{quantile="0.99"} 0.041028634
pool_new_block_sum 7.784695767000002
pool_new_block_count 1418
# HELP pool_process_remote_txs
# TYPE pool_process_remote_txs summary
pool_process_remote_txs{quantile="0.5"} 0.00026041
pool_process_remote_txs{quantile="0.9"} 0.000836163
pool_process_remote_txs{quantile="0.97"} 0.003674852
pool_process_remote_txs{quantile="0.99"} 0.00792353
pool_process_remote_txs_sum 222.03017826299978
pool_process_remote_txs_count 323549
# HELP pool_propagate_new_txs
# TYPE pool_propagate_new_txs summary
pool_propagate_new_txs{quantile="0.5"} 0.000166068
pool_propagate_new_txs{quantile="0.9"} 0.000236169
pool_propagate_new_txs{quantile="0.97"} 0.000314877
pool_propagate_new_txs{quantile="0.99"} 0.000505156
pool_propagate_new_txs_sum 33.44220995600026
pool_propagate_new_txs_count 193640
# HELP pool_propagate_to_new_peer
# TYPE pool_propagate_to_new_peer summary
pool_propagate_to_new_peer{quantile="0.5"} 0.004443281
pool_propagate_to_new_peer{quantile="0.9"} 0.006513069
pool_propagate_to_new_peer{quantile="0.97"} 0.006513069
pool_propagate_to_new_peer{quantile="0.99"} 0.006513069
pool_propagate_to_new_peer_sum 2.5983682590000017
pool_propagate_to_new_peer_count 882
# HELP pool_write_to_db
# TYPE pool_write_to_db summary
pool_write_to_db{quantile="0.5"} 0.023244453
pool_write_to_db{quantile="0.9"} 0.094639672
pool_write_to_db{quantile="0.97"} 0.095315003
pool_write_to_db{quantile="0.99"} 0.095315003
pool_write_to_db_sum 41.972073511999895
pool_write_to_db_count 2159
# HELP pool_write_to_db_bytes
# TYPE pool_write_to_db_bytes gauge
pool_write_to_db_bytes 3.883008e+06
# HELP process_cpu_seconds_total Total user and system CPU time spent in seconds.
# TYPE process_cpu_seconds_total counter
process_cpu_seconds_total 31409.98
# HELP process_max_fds Maximum number of open file descriptors.
# TYPE process_max_fds gauge
process_max_fds 524288
# HELP process_open_fds Number of open file descriptors.
# TYPE process_open_fds gauge
process_open_fds 6965
# HELP process_resident_memory_bytes Resident memory size in bytes.
# TYPE process_resident_memory_bytes gauge
process_resident_memory_bytes 2.9412171776e+10
# HELP process_start_time_seconds Start time of the process since unix epoch in seconds.
# TYPE process_start_time_seconds gauge
process_start_time_seconds 1.7171252653e+09
# HELP process_virtual_memory_bytes Virtual memory size in bytes.
# TYPE process_virtual_memory_bytes gauge
process_virtual_memory_bytes 1.9924772687872e+13
# HELP process_virtual_memory_max_bytes Maximum amount of virtual memory available in bytes.
# TYPE process_virtual_memory_max_bytes gauge
process_virtual_memory_max_bytes 1.8446744073709552e+19
# HELP promhttp_metric_handler_requests_in_flight Current number of scrapes being served.
# TYPE promhttp_metric_handler_requests_in_flight gauge
promhttp_metric_handler_requests_in_flight 1
# HELP promhttp_metric_handler_requests_total Total number of scrapes by HTTP status code.
# TYPE promhttp_metric_handler_requests_total counter
promhttp_metric_handler_requests_total{code="200"} 0
promhttp_metric_handler_requests_total{code="500"} 0
promhttp_metric_handler_requests_total{code="503"} 0
# HELP rpc_duration_seconds
# TYPE rpc_duration_seconds summary
rpc_duration_seconds{method="debug_traceBlockByNumber",success="success",quantile="0.5"} 0.251470495
rpc_duration_seconds{method="debug_traceBlockByNumber",success="success",quantile="0.9"} 0.462843089
rpc_duration_seconds{method="debug_traceBlockByNumber",success="success",quantile="0.97"} 0.566487598
rpc_duration_seconds{method="debug_traceBlockByNumber",success="success",quantile="0.99"} 0.64432017
rpc_duration_seconds_sum{method="debug_traceBlockByNumber",success="success"} 63.26511642199999
rpc_duration_seconds_count{method="debug_traceBlockByNumber",success="success"} 233
rpc_duration_seconds{method="eth_call",success="failure",quantile="0.5"} 0.000264239
rpc_duration_seconds{method="eth_call",success="failure",quantile="0.9"} 0.000435828
rpc_duration_seconds{method="eth_call",success="failure",quantile="0.97"} 0.000566298
rpc_duration_seconds{method="eth_call",success="failure",quantile="0.99"} 0.000908582
rpc_duration_seconds_sum{method="eth_call",success="failure"} 2.382924629999998
rpc_duration_seconds_count{method="eth_call",success="failure"} 7109
rpc_duration_seconds{method="eth_call",success="success",quantile="0.5"} 0.000289541
rpc_duration_seconds{method="eth_call",success="success",quantile="0.9"} 0.000474839
rpc_duration_seconds{method="eth_call",success="success",quantile="0.97"} 0.000837497
rpc_duration_seconds{method="eth_call",success="success",quantile="0.99"} 0.002834341
rpc_duration_seconds_sum{method="eth_call",success="success"} 67.55609726600026
rpc_duration_seconds_count{method="eth_call",success="success"} 101262
rpc_duration_seconds{method="eth_chainId",success="success",quantile="0.5"} 3.7343e-05
rpc_duration_seconds{method="eth_chainId",success="success",quantile="0.9"} 5.5123e-05
rpc_duration_seconds{method="eth_chainId",success="success",quantile="0.97"} 7.608e-05
rpc_duration_seconds{method="eth_chainId",success="success",quantile="0.99"} 0.00012031
rpc_duration_seconds_sum{method="eth_chainId",success="success"} 10.313906366999971
rpc_duration_seconds_count{method="eth_chainId",success="success"} 216740
rpc_duration_seconds{method="eth_getBalance",success="success",quantile="0.5"} 0.000568355
rpc_duration_seconds{method="eth_getBalance",success="success",quantile="0.9"} 0.000989344
rpc_duration_seconds{method="eth_getBalance",success="success",quantile="0.97"} 0.004018818
rpc_duration_seconds{method="eth_getBalance",success="success",quantile="0.99"} 0.005628827
rpc_duration_seconds_sum{method="eth_getBalance",success="success"} 3.122616261999997
rpc_duration_seconds_count{method="eth_getBalance",success="success"} 7315
rpc_duration_seconds{method="eth_getBlockByNumber",success="success",quantile="0.5"} 0.006573431
rpc_duration_seconds{method="eth_getBlockByNumber",success="success",quantile="0.9"} 0.010861762
rpc_duration_seconds{method="eth_getBlockByNumber",success="success",quantile="0.97"} 0.015142079
rpc_duration_seconds{method="eth_getBlockByNumber",success="success",quantile="0.99"} 0.021168075
rpc_duration_seconds_sum{method="eth_getBlockByNumber",success="success"} 1.836215155999999
rpc_duration_seconds_count{method="eth_getBlockByNumber",success="success"} 274
rpc_duration_seconds{method="eth_getBlockReceipts",success="success",quantile="0.5"} 0.014439609
rpc_duration_seconds{method="eth_getBlockReceipts",success="success",quantile="0.9"} 0.025190802
rpc_duration_seconds{method="eth_getBlockReceipts",success="success",quantile="0.97"} 0.03037824
rpc_duration_seconds{method="eth_getBlockReceipts",success="success",quantile="0.99"} 0.037316717
rpc_duration_seconds_sum{method="eth_getBlockReceipts",success="success"} 3.627353174999999
rpc_duration_seconds_count{method="eth_getBlockReceipts",success="success"} 234
rpc_duration_seconds{method="eth_getCode",success="success",quantile="0.5"} 9.8417e-05
rpc_duration_seconds{method="eth_getCode",success="success",quantile="0.9"} 0.000201195
rpc_duration_seconds{method="eth_getCode",success="success",quantile="0.97"} 0.000311493
rpc_duration_seconds{method="eth_getCode",success="success",quantile="0.99"} 0.000463723
rpc_duration_seconds_sum{method="eth_getCode",success="success"} 0.09250182100000001
rpc_duration_seconds_count{method="eth_getCode",success="success"} 749
rpc_duration_seconds{method="eth_getTransactionReceipt",success="success",quantile="0.5"} 0.007869331
rpc_duration_seconds{method="eth_getTransactionReceipt",success="success",quantile="0.9"} 0.007869331
rpc_duration_seconds{method="eth_getTransactionReceipt",success="success",quantile="0.97"} 0.007869331
rpc_duration_seconds{method="eth_getTransactionReceipt",success="success",quantile="0.99"} 0.007869331
rpc_duration_seconds_sum{method="eth_getTransactionReceipt",success="success"} 0.26833929399999995
rpc_duration_seconds_count{method="eth_getTransactionReceipt",success="success"} 59
rpc_duration_seconds{method="web3_clientVersion",success="success",quantile="0.5"} 2.8654e-05
rpc_duration_seconds{method="web3_clientVersion",success="success",quantile="0.9"} 5.765e-05
rpc_duration_seconds{method="web3_clientVersion",success="success",quantile="0.97"} 5.765e-05
rpc_duration_seconds{method="web3_clientVersion",success="success",quantile="0.99"} 5.765e-05
rpc_duration_seconds_sum{method="web3_clientVersion",success="success"} 0.15894310299999997
rpc_duration_seconds_count{method="web3_clientVersion",success="success"} 333
# HELP rpc_failure
# TYPE rpc_failure counter
rpc_failure 7109
# HELP rpc_total
# TYPE rpc_total counter
rpc_total 334308
# HELP sync
# TYPE sync gauge
sync{stage="account_history_index"} 1.9989743e+07
sync{stage="block_hashes"} 1.9989745e+07
sync{stage="bodies"} 1.9986882e+07
sync{stage="bor_heimdall"} 0
sync{stage="call_traces"} 1.9989743e+07
sync{stage="execution"} 1.9989745e+07
sync{stage="finish"} 1.9989743e+07
sync{stage="hash_state"} 1.9989745e+07
sync{stage="headers"} 1.9986882e+07
sync{stage="intermediate_hashes"} 1.9989744e+07
sync{stage="log_index"} 1.9989743e+07
sync{stage="senders"} 1.9989745e+07
sync{stage="snapshots"} 1.9989739e+07
sync{stage="storage_history_index"} 1.9989743e+07
sync{stage="translation"} 0
sync{stage="tx_lookup"} 1.9989743e+07
# HELP tx_dirty
# TYPE tx_dirty gauge
tx_dirty 0
# HELP tx_limit
# TYPE tx_limit gauge
tx_limit 1.34217728e+08
# HELP tx_spill
# TYPE tx_spill gauge
tx_spill 0
# HELP tx_unspill
# TYPE tx_unspill gauge
tx_unspill 0
# HELP txpool_basefee
# TYPE txpool_basefee gauge
txpool_basefee 3039
# HELP txpool_pending
# TYPE txpool_pending gauge
txpool_pending 489
# HELP txpool_queued
# TYPE txpool_queued gauge
txpool_queued 482

Lat logs:

may 31 14:32:34 root-pc erigon[25470]: [INFO] [05-31|14:32:34.476] [txpool] stat                            pending=228 baseFee=2705 queued=464 alloc=11.7GB sys=18.4GB
may 31 14:32:35 root-pc erigon[25470]: [INFO] [05-31|14:32:35.417] [mem] memory stats                       Rss=22.4MB Size=0B Pss=22.4MB SharedClean=2.7KB SharedDirty=0B PrivateClean=4.0MB PrivateDirty=18.5MB Referenced=22.2MB Anonymous=18.4MB Swap=104.5KB alloc=11.7GB sys=18.4GB
may 31 14:32:42 root-pc erigon[25470]: [INFO] [05-31|14:32:42.637] [4/12 Execution] Completed on            block=19989538
may 31 14:32:43 root-pc erigon[25470]: [INFO] [05-31|14:32:43.593] RPC Daemon notified of new headers       from=19989536 to=19989538 amount=2 hash=0xd7b71193e09e3ab156833eeeda9f4f889fe1617367e115788878b91ed33238c9 header sending=17.351µs log sending=621ns
may 31 14:32:43 root-pc erigon[25470]: [INFO] [05-31|14:32:43.593] head updated                             hash=0xd7b71193e09e3ab156833eeeda9f4f889fe1617367e115788878b91ed33238c9 number=19989538
may 31 14:32:54 root-pc erigon[25470]: [INFO] [05-31|14:32:54.585] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
may 31 14:32:55 root-pc erigon[25470]: [INFO] [05-31|14:32:55.023] RPC Daemon notified of new headers       from=19989538 to=19989539 amount=1 hash=0x3858987d61cfe07081c5ae90cb7752565dca981b82cf2a94db1395d3c562d7cd header sending=21.065µs log sending=354ns
may 31 14:32:55 root-pc erigon[25470]: [INFO] [05-31|14:32:55.023] head updated                             hash=0x3858987d61cfe07081c5ae90cb7752565dca981b82cf2a94db1395d3c562d7cd number=19989539
may 31 14:33:05 root-pc erigon[25470]: [INFO] [05-31|14:33:05.621] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
may 31 14:33:05 root-pc erigon[25470]: [INFO] [05-31|14:33:05.975] RPC Daemon notified of new headers       from=19989539 to=19989540 amount=1 hash=0xb2524efaf083bf4dd1906fb746302635e990b3eeaab05cd0bf7d298060902368 header sending=22.668µs log sending=363ns
may 31 14:33:05 root-pc erigon[25470]: [INFO] [05-31|14:33:05.975] head updated                             hash=0xb2524efaf083bf4dd1906fb746302635e990b3eeaab05cd0bf7d298060902368 number=19989540
may 31 14:33:11 root-pc erigon[25470]: [INFO] [05-31|14:33:11.685] P2P                                      app=caplin peers=63
may 31 14:33:46 root-pc erigon[25470]: [INFO] [05-31|14:33:46.266] [4/12 Execution] Completed on            block=19989543
may 31 14:33:47 root-pc erigon[25470]: [INFO] [05-31|14:33:47.710] RPC Daemon notified of new headers       from=19989540 to=19989543 amount=3 hash=0x18c6507653f580dbbf1c9d80e44864b09721c8e6436a13aaedcd3c65445696bf header sending=33.298µs log sending=460ns
may 31 14:33:47 root-pc erigon[25470]: [INFO] [05-31|14:33:47.710] head updated                             hash=0x18c6507653f580dbbf1c9d80e44864b09721c8e6436a13aaedcd3c65445696bf number=19989543
may 31 14:33:55 root-pc erigon[25470]: [INFO] [05-31|14:33:55.739] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
may 31 14:33:56 root-pc erigon[25470]: [INFO] [05-31|14:33:56.325] RPC Daemon notified of new headers       from=19989543 to=19989544 amount=1 hash=0x0ef7fafdfc98949afc1aba6a182808c570acb8ea80799faab36bfbb384fcdba3 header sending=16.604µs log sending=625ns
may 31 14:33:56 root-pc erigon[25470]: [INFO] [05-31|14:33:56.325] head updated                             hash=0x0ef7fafdfc98949afc1aba6a182808c570acb8ea80799faab36bfbb384fcdba3 number=19989544
may 31 14:34:09 root-pc erigon[25470]: [INFO] [05-31|14:34:09.146] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
may 31 14:34:09 root-pc erigon[25470]: [INFO] [05-31|14:34:09.625] RPC Daemon notified of new headers       from=19989544 to=19989545 amount=1 hash=0x4a8030d33d89fc71d50730050f086cc138786630bb488f9b24a272e4cef4538c header sending=19.542µs log sending=404ns
may 31 14:34:09 root-pc erigon[25470]: [INFO] [05-31|14:34:09.625] head updated                             hash=0x4a8030d33d89fc71d50730050f086cc138786630bb488f9b24a272e4cef4538c number=19989545
may 31 14:34:11 root-pc erigon[25470]: [INFO] [05-31|14:34:11.684] P2P                                      app=caplin peers=61
may 31 14:34:17 root-pc erigon[25470]: [INFO] [05-31|14:34:17.286] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
may 31 14:34:17 root-pc erigon[25470]: [INFO] [05-31|14:34:17.691] RPC Daemon notified of new headers       from=19989545 to=19989546 amount=1 hash=0xb4ee9f10baa8bc887e64ffea5c3be9f174ee1311c2d8b0026566fed320fcc2da header sending=19.421µs log sending=418ns
may 31 14:34:17 root-pc erigon[25470]: [INFO] [05-31|14:34:17.691] head updated                             hash=0xb4ee9f10baa8bc887e64ffea5c3be9f174ee1311c2d8b0026566fed320fcc2da number=19989546
may 31 14:34:31 root-pc erigon[25470]: [INFO] [05-31|14:34:31.857] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
may 31 14:34:32 root-pc erigon[25470]: [INFO] [05-31|14:34:32.337] RPC Daemon notified of new headers       from=19989546 to=19989547 amount=1 hash=0xbe63b891a74582a264c06c47b2872f2a4ab217d3dd84d80caa90983f951f4d71 header sending=16.638µs log sending=389ns
may 31 14:34:32 root-pc erigon[25470]: [INFO] [05-31|14:34:32.337] head updated                             hash=0xbe63b891a74582a264c06c47b2872f2a4ab217d3dd84d80caa90983f951f4d71 number=19989547
may 31 14:34:44 root-pc erigon[25470]: [INFO] [05-31|14:34:44.203] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
may 31 14:34:44 root-pc erigon[25470]: [INFO] [05-31|14:34:44.745] RPC Daemon notified of new headers       from=19989547 to=19989548 amount=1 hash=0xe30925719af0db2eb075454e7165e3839bf4e46f7bc19f44264043205213650d header sending=45.359µs log sending=614ns
may 31 14:34:44 root-pc erigon[25470]: [INFO] [05-31|14:34:44.746] head updated                             hash=0xe30925719af0db2eb075454e7165e3839bf4e46f7bc19f44264043205213650d number=19989548
may 31 14:35:11 root-pc erigon[25470]: [INFO] [05-31|14:35:11.684] P2P                                      app=caplin peers=65
may 31 14:35:33 root-pc erigon[25470]: [INFO] [05-31|14:35:33.876] [p2p] GoodPeers                          eth68=32 eth67=6
may 31 14:35:34 root-pc erigon[25470]: [INFO] [05-31|14:35:34.476] [txpool] stat                            pending=657 baseFee=2738 queued=476 alloc=10.1GB sys=18.4GB
may 31 14:35:35 root-pc erigon[25470]: [INFO] [05-31|14:35:35.345] [mem] memory stats                       Rss=21.7MB Size=0B Pss=21.7MB SharedClean=2.7KB SharedDirty=0B PrivateClean=2.5MB PrivateDirty=19.2MB Referenced=21.5MB Anonymous=19.1MB Swap=104.5KB alloc=10.1GB sys=18.4GB
may 31 14:35:46 root-pc erigon[25470]: [INFO] [05-31|14:35:46.276] [4/12 Execution] Completed on            block=19989553
may 31 14:35:49 root-pc erigon[25470]: [INFO] [05-31|14:35:49.201] RPC Daemon notified of new headers       from=19989548 to=19989553 amount=5 hash=0xbe279395d4a0fc23fdcfc54913193412cafbbf8a62e9cd8f42eef79fc6422162 header sending=42.007µs log sending=399ns
may 31 14:35:49 root-pc erigon[25470]: [INFO] [05-31|14:35:49.201] head updated                             hash=0xbe279395d4a0fc23fdcfc54913193412cafbbf8a62e9cd8f42eef79fc6422162 number=19989553
may 31 14:36:11 root-pc erigon[25470]: [INFO] [05-31|14:36:11.684] P2P                                      app=caplin peers=61
may 31 14:36:57 root-pc erigon[25470]: [INFO] [05-31|14:36:57.053] [4/12 Execution] Completed on            block=19989559
may 31 14:37:00 root-pc erigon[25470]: [INFO] [05-31|14:37:00.429] RPC Daemon notified of new headers       from=19989553 to=19989559 amount=6 hash=0x5cf82abec74fd0ebe7478693b0d68c81b620649cb3b0e3c0688d51bd267cde98 header sending=22.681µs log sending=360ns
may 31 14:37:00 root-pc erigon[25470]: [INFO] [05-31|14:37:00.429] head updated                             hash=0x5cf82abec74fd0ebe7478693b0d68c81b620649cb3b0e3c0688d51bd267cde98 number=19989559
may 31 14:37:11 root-pc erigon[25470]: [INFO] [05-31|14:37:11.684] P2P                                      app=caplin peers=57
may 31 14:37:55 root-pc erigon[25470]: [INFO] [05-31|14:37:55.505] [4/12 Execution] Completed on            block=19989564
may 31 14:37:58 root-pc erigon[25470]: [INFO] [05-31|14:37:58.215] RPC Daemon notified of new headers       from=19989559 to=19989564 amount=5 hash=0xf403fc093db551834ea55e37a7a66624b69822e50e6569d4949b0836500aac97 header sending=54.01µs log sending=963ns
may 31 14:37:58 root-pc erigon[25470]: [INFO] [05-31|14:37:58.215] head updated                             hash=0xf403fc093db551834ea55e37a7a66624b69822e50e6569d4949b0836500aac97 number=19989564
may 31 14:38:08 root-pc erigon[25470]: [INFO] [05-31|14:38:08.598] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
may 31 14:38:09 root-pc erigon[25470]: [INFO] [05-31|14:38:09.156] RPC Daemon notified of new headers       from=19989564 to=19989565 amount=1 hash=0x66d3571b5e03ff782ab2a1af2aca5b32f47acb519b1037ab3dba6992d2676ad8 header sending=18.654µs log sending=695ns
may 31 14:38:09 root-pc erigon[25470]: [INFO] [05-31|14:38:09.156] head updated                             hash=0x66d3571b5e03ff782ab2a1af2aca5b32f47acb519b1037ab3dba6992d2676ad8 number=19989565
may 31 14:38:11 root-pc erigon[25470]: [INFO] [05-31|14:38:11.684] P2P                                      app=caplin peers=65
may 31 14:38:17 root-pc erigon[25470]: [INFO] [05-31|14:38:17.450] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
may 31 14:38:18 root-pc erigon[25470]: [INFO] [05-31|14:38:18.103] RPC Daemon notified of new headers       from=19989565 to=19989566 amount=1 hash=0x99c9ad369880ca4db11745346eb78500ffc50fc6cea8dc5726c7ff1599d8aa1c header sending=20.296µs log sending=391ns
may 31 14:38:18 root-pc erigon[25470]: [INFO] [05-31|14:38:18.103] head updated                             hash=0x99c9ad369880ca4db11745346eb78500ffc50fc6cea8dc5726c7ff1599d8aa1c number=19989566
may 31 14:38:33 root-pc erigon[25470]: [INFO] [05-31|14:38:33.876] [p2p] GoodPeers                          eth68=33 eth67=6
may 31 14:38:34 root-pc erigon[25470]: [INFO] [05-31|14:38:34.475] [txpool] stat                            pending=241 baseFee=2827 queued=466 alloc=7.3GB sys=18.4GB
may 31 14:38:35 root-pc erigon[25470]: [INFO] [05-31|14:38:35.324] [mem] memory stats                       Rss=20.5MB Size=0B Pss=20.5MB SharedClean=2.7KB SharedDirty=0B PrivateClean=2.6MB PrivateDirty=17.8MB Referenced=20.3MB Anonymous=17.8MB Swap=104.5KB alloc=7.3GB sys=18.4GB
may 31 14:38:46 root-pc erigon[25470]: [INFO] [05-31|14:38:46.097] [4/12 Execution] Completed on            block=19989568
may 31 14:38:46 root-pc erigon[25470]: [INFO] [05-31|14:38:46.889] RPC Daemon notified of new headers       from=19989566 to=19989568 amount=2 hash=0x10b35bac41f94b3f519e449e25e1c5aaaf522f1a07c7c64e171cbc696b6dac8e header sending=18.846µs log sending=349ns
may 31 14:38:46 root-pc erigon[25470]: [INFO] [05-31|14:38:46.889] head updated                             hash=0x10b35bac41f94b3f519e449e25e1c5aaaf522f1a07c7c64e171cbc696b6dac8e number=19989568
may 31 14:38:57 root-pc erigon[25470]: [INFO] [05-31|14:38:57.130] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
may 31 14:38:57 root-pc erigon[25470]: [INFO] [05-31|14:38:57.999] RPC Daemon notified of new headers       from=19989568 to=19989569 amount=1 hash=0x96d634095067d8e7f1acb55835bbe8cc03e84676902403470fb766c107a71634 header sending=20.812µs log sending=383ns
may 31 14:38:57 root-pc erigon[25470]: [INFO] [05-31|14:38:57.999] head updated                             hash=0x96d634095067d8e7f1acb55835bbe8cc03e84676902403470fb766c107a71634 number=19989569
may 31 14:39:07 root-pc erigon[25470]: [INFO] [05-31|14:39:07.245] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
may 31 14:39:07 root-pc erigon[25470]: [INFO] [05-31|14:39:07.744] RPC Daemon notified of new headers       from=19989569 to=19989570 amount=1 hash=0xe033b460e4a7d6d44d74f4f7c7481a26a6b7585f9ede12ab4304768482e76b0a header sending=18.783µs log sending=411ns
may 31 14:39:07 root-pc erigon[25470]: [INFO] [05-31|14:39:07.744] head updated                             hash=0xe033b460e4a7d6d44d74f4f7c7481a26a6b7585f9ede12ab4304768482e76b0a number=19989570
may 31 14:39:11 root-pc erigon[25470]: [INFO] [05-31|14:39:11.684] P2P                                      app=caplin peers=65
may 31 14:39:19 root-pc erigon[25470]: [INFO] [05-31|14:39:19.109] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
may 31 14:39:19 root-pc erigon[25470]: [INFO] [05-31|14:39:19.675] RPC Daemon notified of new headers       from=19989570 to=19989571 amount=1 hash=0x0e85bf43f44ee42e8af9a0e64eaf9b6898f28d27923a1e160ebe37ee35609b53 header sending=17.744µs log sending=450ns
may 31 14:39:19 root-pc erigon[25470]: [INFO] [05-31|14:39:19.675] head updated                             hash=0x0e85bf43f44ee42e8af9a0e64eaf9b6898f28d27923a1e160ebe37ee35609b53 number=19989571
may 31 14:39:29 root-pc erigon[25470]: [INFO] [05-31|14:39:29.658] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
may 31 14:39:30 root-pc erigon[25470]: [INFO] [05-31|14:39:30.044] RPC Daemon notified of new headers       from=19989571 to=19989572 amount=1 hash=0x0a4668ae9463783d6ba416add3656cc9bf98a3b54a70621d49e51d0b92dc4230 header sending=20.676µs log sending=432ns
may 31 14:39:30 root-pc erigon[25470]: [INFO] [05-31|14:39:30.045] head updated                             hash=0x0a4668ae9463783d6ba416add3656cc9bf98a3b54a70621d49e51d0b92dc4230 number=19989572
may 31 14:40:11 root-pc erigon[25470]: [INFO] [05-31|14:40:11.683] P2P                                      app=caplin peers=66
may 31 14:40:43 root-pc erigon[25470]: [INFO] [05-31|14:40:43.585] [4/12 Execution] Completed on            block=19989578
may 31 14:40:46 root-pc erigon[25470]: [INFO] [05-31|14:40:46.635] RPC Daemon notified of new headers       from=19989572 to=19989578 amount=6 hash=0x2a8d790b03724466441a6ab0a405c1f06865bc5073beb7a378fbed7b0ff41aa4 header sending=24.095µs log sending=588ns
may 31 14:40:46 root-pc erigon[25470]: [INFO] [05-31|14:40:46.636] head updated                             hash=0x2a8d790b03724466441a6ab0a405c1f06865bc5073beb7a378fbed7b0ff41aa4 number=19989578
may 31 14:40:55 root-pc erigon[25470]: [INFO] [05-31|14:40:55.690] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
may 31 14:40:56 root-pc erigon[25470]: [INFO] [05-31|14:40:56.199] RPC Daemon notified of new headers       from=19989578 to=19989579 amount=1 hash=0x9e4c5b87ec6b5b768ad5730510540068df73278f997b1ef09a8bc4c7dc63f86a header sending=18.844µs log sending=738ns
may 31 14:40:56 root-pc erigon[25470]: [INFO] [05-31|14:40:56.200] head updated                             hash=0x9e4c5b87ec6b5b768ad5730510540068df73278f997b1ef09a8bc4c7dc63f86a number=19989579
may 31 14:41:08 root-pc erigon[25470]: [INFO] [05-31|14:41:08.090] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
may 31 14:41:08 root-pc erigon[25470]: [INFO] [05-31|14:41:08.623] RPC Daemon notified of new headers       from=19989579 to=19989580 amount=1 hash=0x8c14c9f9af2aca7ddc41e17f6f6784ee8f1e82f5b798a7da7bfa1b4f653a9d33 header sending=39.789µs log sending=825ns
may 31 14:41:08 root-pc erigon[25470]: [INFO] [05-31|14:41:08.623] head updated                             hash=0x8c14c9f9af2aca7ddc41e17f6f6784ee8f1e82f5b798a7da7bfa1b4f653a9d33 number=19989580
may 31 14:41:11 root-pc erigon[25470]: [INFO] [05-31|14:41:11.684] P2P                                      app=caplin peers=59
may 31 14:41:22 root-pc erigon[25470]: [INFO] [05-31|14:41:22.482] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
may 31 14:41:22 root-pc erigon[25470]: [INFO] [05-31|14:41:22.968] RPC Daemon notified of new headers       from=19989580 to=19989581 amount=1 hash=0x143162c8f0657f7abd798910d13c60a8293c028e5835031e87535aed79f6ca58 header sending=25.096µs log sending=378ns
may 31 14:41:22 root-pc erigon[25470]: [INFO] [05-31|14:41:22.968] head updated                             hash=0x143162c8f0657f7abd798910d13c60a8293c028e5835031e87535aed79f6ca58 number=19989581
may 31 14:41:32 root-pc erigon[25470]: [INFO] [05-31|14:41:32.886] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
may 31 14:41:33 root-pc erigon[25470]: [INFO] [05-31|14:41:33.678] RPC Daemon notified of new headers       from=19989581 to=19989582 amount=1 hash=0xf14577d8b9df45a5f9a1b8d1adb43916134fce9e4750a49dfafc91ab4d9f74a6 header sending=18.307µs log sending=351ns
may 31 14:41:33 root-pc erigon[25470]: [INFO] [05-31|14:41:33.678] head updated                             hash=0xf14577d8b9df45a5f9a1b8d1adb43916134fce9e4750a49dfafc91ab4d9f74a6 number=19989582
may 31 14:41:33 root-pc erigon[25470]: [INFO] [05-31|14:41:33.876] [p2p] GoodPeers                          eth68=33 eth67=6
may 31 14:41:34 root-pc erigon[25470]: [INFO] [05-31|14:41:34.475] [txpool] stat                            pending=133 baseFee=2758 queued=470 alloc=7.0GB sys=18.4GB
may 31 14:41:35 root-pc erigon[25470]: [INFO] [05-31|14:41:35.335] [mem] memory stats                       Rss=21.6MB Size=0B Pss=21.6MB SharedClean=2.7KB SharedDirty=0B PrivateClean=3.2MB PrivateDirty=18.4MB Referenced=21.4MB Anonymous=18.4MB Swap=104.4KB alloc=7.1GB sys=18.4GB
may 31 14:42:10 root-pc erigon[25470]: [INFO] [05-31|14:42:10.042] [4/12 Execution] Completed on            block=19989585
may 31 14:42:11 root-pc erigon[25470]: [INFO] [05-31|14:42:11.350] RPC Daemon notified of new headers       from=19989582 to=19989585 amount=3 hash=0x7c87588521ad4aaf528de9c36182e27d595947eb3ec0b73987c9206244aa0f0e header sending=19.852µs log sending=379ns
may 31 14:42:11 root-pc erigon[25470]: [INFO] [05-31|14:42:11.350] head updated                             hash=0x7c87588521ad4aaf528de9c36182e27d595947eb3ec0b73987c9206244aa0f0e number=19989585
may 31 14:42:11 root-pc erigon[25470]: [INFO] [05-31|14:42:11.684] P2P                                      app=caplin peers=56
may 31 14:42:21 root-pc erigon[25470]: [INFO] [05-31|14:42:21.601] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
may 31 14:42:22 root-pc erigon[25470]: [INFO] [05-31|14:42:22.303] RPC Daemon notified of new headers       from=19989585 to=19989586 amount=1 hash=0xd2ecd1801de2f3e9a7fc99403a45f18c2e82465713caa7c4be1c40a142d63793 header sending=16.9µs log sending=415ns
may 31 14:42:22 root-pc erigon[25470]: [INFO] [05-31|14:42:22.303] head updated                             hash=0xd2ecd1801de2f3e9a7fc99403a45f18c2e82465713caa7c4be1c40a142d63793 number=19989586
may 31 14:42:46 root-pc erigon[25470]: [INFO] [05-31|14:42:46.635] [4/12 Execution] Completed on            block=19989588
may 31 14:42:47 root-pc erigon[25470]: [INFO] [05-31|14:42:47.664] RPC Daemon notified of new headers       from=19989586 to=19989588 amount=2 hash=0x4dd8c82cb56d412f98cee2bbce536bea32fd6eff2431940f847063a3bf19a156 header sending=24.768µs log sending=409ns
may 31 14:42:47 root-pc erigon[25470]: [INFO] [05-31|14:42:47.664] head updated                             hash=0x4dd8c82cb56d412f98cee2bbce536bea32fd6eff2431940f847063a3bf19a156 number=19989588
may 31 14:42:56 root-pc erigon[25470]: [INFO] [05-31|14:42:56.566] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
may 31 14:42:57 root-pc erigon[25470]: [INFO] [05-31|14:42:57.169] RPC Daemon notified of new headers       from=19989588 to=19989589 amount=1 hash=0x42d6d11afd10ee6cf29ea2aa8d510138fd77565e6fe001c20a8951e3461dcb19 header sending=18.846µs log sending=386ns
may 31 14:42:57 root-pc erigon[25470]: [INFO] [05-31|14:42:57.169] head updated                             hash=0x42d6d11afd10ee6cf29ea2aa8d510138fd77565e6fe001c20a8951e3461dcb19 number=19989589
may 31 14:43:05 root-pc erigon[25470]: [INFO] [05-31|14:43:05.019] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
may 31 14:43:05 root-pc erigon[25470]: [INFO] [05-31|14:43:05.458] RPC Daemon notified of new headers       from=19989589 to=19989590 amount=1 hash=0x0f47662f32170348117c0d5400960ef220a50b7c02ea049501d4859850f4d821 header sending=17.466µs log sending=358ns
may 31 14:43:05 root-pc erigon[25470]: [INFO] [05-31|14:43:05.458] head updated                             hash=0x0f47662f32170348117c0d5400960ef220a50b7c02ea049501d4859850f4d821 number=19989590
may 31 14:43:11 root-pc erigon[25470]: [INFO] [05-31|14:43:11.684] P2P                                      app=caplin peers=57
may 31 14:43:22 root-pc erigon[25470]: [INFO] [05-31|14:43:22.494] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
may 31 14:43:23 root-pc erigon[25470]: [INFO] [05-31|14:43:23.287] RPC Daemon notified of new headers       from=19989590 to=19989591 amount=1 hash=0xa0607c5eae5f52a1b37d6ecb4df6f372eb9bd8222fce0c5d8e032b6c98eba4c0 header sending=20.097µs log sending=372ns
may 31 14:43:23 root-pc erigon[25470]: [INFO] [05-31|14:43:23.287] head updated                             hash=0xa0607c5eae5f52a1b37d6ecb4df6f372eb9bd8222fce0c5d8e032b6c98eba4c0 number=19989591
may 31 14:44:11 root-pc erigon[25470]: [INFO] [05-31|14:44:11.684] P2P                                      app=caplin peers=64
may 31 14:44:33 root-pc erigon[25470]: [INFO] [05-31|14:44:33.876] [p2p] GoodPeers                          eth68=33 eth67=6
may 31 14:44:34 root-pc erigon[25470]: [INFO] [05-31|14:44:34.476] [txpool] stat                            pending=873 baseFee=2813 queued=474 alloc=11.9GB sys=18.4GB
may 31 14:44:35 root-pc erigon[25470]: [INFO] [05-31|14:44:35.518] [mem] memory stats                       Rss=22.3MB Size=0B Pss=22.3MB SharedClean=2.7KB SharedDirty=0B PrivateClean=3.5MB PrivateDirty=18.8MB Referenced=22.1MB Anonymous=18.8MB Swap=104.4KB alloc=12.0GB sys=18.4GB
may 31 14:44:47 root-pc erigon[25470]: [INFO] [05-31|14:44:47.206] [4/12 Execution] Completed on            block=19989598
may 31 14:44:50 root-pc erigon[25470]: [INFO] [05-31|14:44:50.435] RPC Daemon notified of new headers       from=19989591 to=19989598 amount=7 hash=0xaf5d633026f9a325339956f2f88aadbe25a69fb0a8c2c6cbb5007134a2279338 header sending=41.568µs log sending=669ns
may 31 14:44:50 root-pc erigon[25470]: [INFO] [05-31|14:44:50.436] head updated                             hash=0xaf5d633026f9a325339956f2f88aadbe25a69fb0a8c2c6cbb5007134a2279338 number=19989598
may 31 14:45:11 root-pc erigon[25470]: [INFO] [05-31|14:45:11.683] P2P                                      app=caplin peers=64
may 31 14:45:34 root-pc erigon[25470]: [INFO] [05-31|14:45:34.440] [4/12 Execution] Completed on            block=19989602
may 31 14:45:36 root-pc erigon[25470]: [INFO] [05-31|14:45:36.429] RPC Daemon notified of new headers       from=19989598 to=19989602 amount=4 hash=0x9b9685271f30d2e8748e71ef99a3ec195eb979323e5409f6e66344073200c693 header sending=54.672µs log sending=654ns
may 31 14:45:36 root-pc erigon[25470]: [INFO] [05-31|14:45:36.429] head updated                             hash=0x9b9685271f30d2e8748e71ef99a3ec195eb979323e5409f6e66344073200c693 number=19989602
may 31 14:46:09 root-pc erigon[25470]: [INFO] [05-31|14:46:09.450] [4/12 Execution] Completed on            block=19989605
may 31 14:46:11 root-pc erigon[25470]: [INFO] [05-31|14:46:11.058] RPC Daemon notified of new headers       from=19989602 to=19989605 amount=3 hash=0x83a996b819282119f331a00d632af0e6c41b6de5ff4e24fe8339fe84f0597ba2 header sending=22.694µs log sending=413ns
may 31 14:46:11 root-pc erigon[25470]: [INFO] [05-31|14:46:11.058] head updated                             hash=0x83a996b819282119f331a00d632af0e6c41b6de5ff4e24fe8339fe84f0597ba2 number=19989605
may 31 14:46:11 root-pc erigon[25470]: [INFO] [05-31|14:46:11.684] P2P                                      app=caplin peers=61
may 31 14:46:18 root-pc erigon[25470]: [INFO] [05-31|14:46:18.132] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
may 31 14:46:18 root-pc erigon[25470]: [INFO] [05-31|14:46:18.748] RPC Daemon notified of new headers       from=19989605 to=19989606 amount=1 hash=0x63334bbdc88f7cb327591b28d47bce46fccda1ef68532000a96b129574a9fcbd header sending=19.369µs log sending=419ns
may 31 14:46:18 root-pc erigon[25470]: [INFO] [05-31|14:46:18.748] head updated                             hash=0x63334bbdc88f7cb327591b28d47bce46fccda1ef68532000a96b129574a9fcbd number=19989606
may 31 14:46:33 root-pc erigon[25470]: [INFO] [05-31|14:46:33.315] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
may 31 14:46:33 root-pc erigon[25470]: [INFO] [05-31|14:46:33.469] RPC Daemon notified of new headers       from=19989606 to=19989607 amount=1 hash=0xf7c62fb411cc8891f9fd02ba9ed0166442752416d481c544d9cfe3efe6937b3d header sending=20.694µs log sending=405ns
may 31 14:46:33 root-pc erigon[25470]: [INFO] [05-31|14:46:33.469] head updated                             hash=0xf7c62fb411cc8891f9fd02ba9ed0166442752416d481c544d9cfe3efe6937b3d number=19989607
may 31 14:46:43 root-pc erigon[25470]: [INFO] [05-31|14:46:43.895] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
may 31 14:46:44 root-pc erigon[25470]: [INFO] [05-31|14:46:44.031] RPC Daemon notified of new headers       from=19989607 to=19989608 amount=1 hash=0xcd15a4643640037a2faf70d44514b5dc63f5425eeca35ae4a49a946adc000b5c header sending=35.629µs log sending=393ns
may 31 14:46:44 root-pc erigon[25470]: [INFO] [05-31|14:46:44.031] head updated                             hash=0xcd15a4643640037a2faf70d44514b5dc63f5425eeca35ae4a49a946adc000b5c number=19989608
may 31 14:46:55 root-pc erigon[25470]: [INFO] [05-31|14:46:55.182] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
may 31 14:46:56 root-pc erigon[25470]: [INFO] [05-31|14:46:56.238] RPC Daemon notified of new headers       from=19989608 to=19989609 amount=1 hash=0x7917d952ecfff83260de7e24f7a19b84c1207de2d7b4b10ce14adb5fa75b7154 header sending=19.314µs log sending=378ns
may 31 14:46:56 root-pc erigon[25470]: [INFO] [05-31|14:46:56.238] head updated                             hash=0x7917d952ecfff83260de7e24f7a19b84c1207de2d7b4b10ce14adb5fa75b7154 number=19989609
may 31 14:47:06 root-pc erigon[25470]: [INFO] [05-31|14:47:06.313] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
may 31 14:47:07 root-pc erigon[25470]: [INFO] [05-31|14:47:07.159] RPC Daemon notified of new headers       from=19989609 to=19989610 amount=1 hash=0x64d4a06f75b0a1b0f3b2557242eccbc790edfef663ab4413f76201f675b3b79e header sending=21.076µs log sending=391ns
may 31 14:47:07 root-pc erigon[25470]: [INFO] [05-31|14:47:07.159] head updated                             hash=0x64d4a06f75b0a1b0f3b2557242eccbc790edfef663ab4413f76201f675b3b79e number=19989610
may 31 14:47:11 root-pc erigon[25470]: [INFO] [05-31|14:47:11.684] P2P                                      app=caplin peers=61
may 31 14:47:33 root-pc erigon[25470]: [INFO] [05-31|14:47:33.876] [p2p] GoodPeers                          eth68=33 eth67=6
may 31 14:47:34 root-pc erigon[25470]: [INFO] [05-31|14:47:34.480] [txpool] stat                            pending=397 baseFee=2756 queued=472 alloc=8.5GB sys=18.4GB
may 31 14:47:35 root-pc erigon[25470]: [INFO] [05-31|14:47:35.451] [mem] memory stats                       Rss=21.9MB Size=0B Pss=21.9MB SharedClean=2.7KB SharedDirty=0B PrivateClean=3.5MB PrivateDirty=18.4MB Referenced=21.7MB Anonymous=18.4MB Swap=104.4KB alloc=8.6GB sys=18.4GB
may 31 14:47:45 root-pc erigon[25470]: [INFO] [05-31|14:47:45.926] [4/12 Execution] Completed on            block=19989613
may 31 14:47:47 root-pc erigon[25470]: [INFO] [05-31|14:47:47.461] RPC Daemon notified of new headers       from=19989610 to=19989613 amount=3 hash=0x84085ec031323e932dd98461978f457d2ed3e2ebc133e7bd244c903dc4a829d2 header sending=26.528µs log sending=419ns
may 31 14:47:47 root-pc erigon[25470]: [INFO] [05-31|14:47:47.461] head updated                             hash=0x84085ec031323e932dd98461978f457d2ed3e2ebc133e7bd244c903dc4a829d2 number=19989613
may 31 14:47:59 root-pc erigon[25470]: [INFO] [05-31|14:47:59.455] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
may 31 14:47:59 root-pc erigon[25470]: [INFO] [05-31|14:47:59.912] RPC Daemon notified of new headers       from=19989613 to=19989614 amount=1 hash=0x17106bbccc1c711c745aadd22d7c52718c1393c7e4e42828ff6b814545eede36 header sending=25.142µs log sending=438ns
may 31 14:47:59 root-pc erigon[25470]: [INFO] [05-31|14:47:59.913] head updated                             hash=0x17106bbccc1c711c745aadd22d7c52718c1393c7e4e42828ff6b814545eede36 number=19989614
may 31 14:48:11 root-pc erigon[25470]: [INFO] [05-31|14:48:11.684] P2P                                      app=caplin peers=65
may 31 14:48:20 root-pc erigon[25470]: [INFO] [05-31|14:48:20.906] [4/12 Execution] Completed on            block=19989616
may 31 14:48:22 root-pc erigon[25470]: [INFO] [05-31|14:48:22.102] RPC Daemon notified of new headers       from=19989614 to=19989616 amount=2 hash=0xb6f41cc74c86961ab99437df09a42b1adfdc4c81631f6a4e885f56e26270a910 header sending=20.418µs log sending=621ns
may 31 14:48:22 root-pc erigon[25470]: [INFO] [05-31|14:48:22.102] head updated                             hash=0xb6f41cc74c86961ab99437df09a42b1adfdc4c81631f6a4e885f56e26270a910 number=19989616
may 31 14:48:32 root-pc erigon[25470]: [INFO] [05-31|14:48:32.674] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
may 31 14:48:33 root-pc erigon[25470]: [INFO] [05-31|14:48:33.528] RPC Daemon notified of new headers       from=19989616 to=19989617 amount=1 hash=0xec121beab83b8d0e646b3ec1b97609ffda603ac44bd6bf49e891bb6304aa0c14 header sending=18.1µs log sending=425ns
may 31 14:48:33 root-pc erigon[25470]: [INFO] [05-31|14:48:33.528] head updated                             hash=0xec121beab83b8d0e646b3ec1b97609ffda603ac44bd6bf49e891bb6304aa0c14 number=19989617
may 31 14:48:46 root-pc erigon[25470]: [INFO] [05-31|14:48:46.625] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
may 31 14:48:47 root-pc erigon[25470]: [INFO] [05-31|14:48:47.224] RPC Daemon notified of new headers       from=19989617 to=19989618 amount=1 hash=0x8a3e2bafa59a94d5e78e74478f14ac0f57e9037c068f7df4ba99526f03850509 header sending=21.449µs log sending=405ns
may 31 14:48:47 root-pc erigon[25470]: [INFO] [05-31|14:48:47.225] head updated                             hash=0x8a3e2bafa59a94d5e78e74478f14ac0f57e9037c068f7df4ba99526f03850509 number=19989618
may 31 14:49:11 root-pc erigon[25470]: [INFO] [05-31|14:49:11.684] P2P                                      app=caplin peers=61
may 31 14:50:11 root-pc erigon[25470]: [INFO] [05-31|14:50:11.685] P2P                                      app=caplin peers=64
may 31 14:50:33 root-pc erigon[25470]: [INFO] [05-31|14:50:33.875] [p2p] GoodPeers                          eth68=33 eth67=6
may 31 14:50:34 root-pc erigon[25470]: [INFO] [05-31|14:50:34.476] [txpool] stat                            pending=1476 baseFee=2742 queued=501 alloc=7.2GB sys=18.4GB
may 31 14:50:35 root-pc erigon[25470]: [INFO] [05-31|14:50:35.312] [mem] memory stats                       Rss=19.1MB Size=0B Pss=19.0MB SharedClean=2.7KB SharedDirty=0B PrivateClean=1.6MB PrivateDirty=17.4MB Referenced=18.9MB Anonymous=17.4MB Swap=104.2KB alloc=7.2GB sys=18.4GB
may 31 14:51:11 root-pc erigon[25470]: [INFO] [05-31|14:51:11.684] P2P                                      app=caplin peers=65
may 31 14:52:11 root-pc erigon[25470]: [INFO] [05-31|14:52:11.684] P2P                                      app=caplin peers=60
may 31 14:52:59 root-pc erigon[25470]: [INFO] [05-31|14:52:59.008] [3/12 Senders] Started                   from=19989618 to=19989639
may 31 14:52:59 root-pc erigon[25470]: [INFO] [05-31|14:52:59.048] [4/12 Execution] Blocks execution        from=19989618 to=19989639
may 31 14:53:00 root-pc erigon[25470]: [INFO] [05-31|14:53:00.804] [4/12 Execution] Completed on            block=19989639
may 31 14:53:00 root-pc erigon[25470]: [INFO] [05-31|14:53:00.804] [5/12 HashState] Promoting plain state   from=19989618 to=19989639
may 31 14:53:00 root-pc erigon[25470]: [INFO] [05-31|14:53:00.804] [5/12 HashState] Incremental promotion   from=19989618 to=19989639 codes=true csbucket=AccountChangeSet
may 31 14:53:01 root-pc erigon[25470]: [INFO] [05-31|14:53:01.187] [5/12 HashState] Incremental promotion   from=19989618 to=19989639 codes=false csbucket=AccountChangeSet
may 31 14:53:01 root-pc erigon[25470]: [INFO] [05-31|14:53:01.318] [5/12 HashState] Incremental promotion   from=19989618 to=19989639 codes=false csbucket=StorageChangeSet
may 31 14:53:01 root-pc erigon[25470]: [INFO] [05-31|14:53:01.760] [6/12 IntermediateHashes] Generating intermediate hashes from=19989618 to=19989639
may 31 14:53:03 root-pc erigon[25470]: [INFO] [05-31|14:53:03.183] [7/12 CallTraces] Pruned call trace intermediate table from=19899618 to=19899638
may 31 14:53:11 root-pc erigon[25470]: [INFO] [05-31|14:53:11.503] RPC Daemon notified of new headers       from=19989618 to=19989639 amount=21 hash=0x29bc5aed0760da78b7c11f773106c8bfdaee796aebf32a85a1176e7170b25ebe header sending=44.989µs log sending=661ns
may 31 14:53:11 root-pc erigon[25470]: [INFO] [05-31|14:53:11.503] head updated                             hash=0x29bc5aed0760da78b7c11f773106c8bfdaee796aebf32a85a1176e7170b25ebe number=19989639
may 31 14:53:11 root-pc erigon[25470]: [INFO] [05-31|14:53:11.684] P2P                                      app=caplin peers=61
may 31 14:53:33 root-pc erigon[25470]: [INFO] [05-31|14:53:33.876] [p2p] GoodPeers                          eth68=33 eth67=6
may 31 14:53:34 root-pc erigon[25470]: [INFO] [05-31|14:53:34.477] [txpool] stat                            pending=441 baseFee=2783 queued=508 alloc=12.5GB sys=18.4GB
may 31 14:53:35 root-pc erigon[25470]: [INFO] [05-31|14:53:35.293] [mem] memory stats                       Rss=19.7MB Size=0B Pss=19.7MB SharedClean=2.7KB SharedDirty=0B PrivateClean=2.3MB PrivateDirty=17.3MB Referenced=19.4MB Anonymous=17.3MB Swap=104.2KB alloc=12.5GB sys=18.4GB
may 31 14:54:11 root-pc erigon[25470]: [INFO] [05-31|14:54:11.684] P2P                                      app=caplin peers=65
may 31 14:54:22 root-pc erigon[25470]: [INFO] [05-31|14:54:22.294] [4/12 Execution] Completed on            block=19989646
may 31 14:54:25 root-pc erigon[25470]: [INFO] [05-31|14:54:25.685] RPC Daemon notified of new headers       from=19989639 to=19989646 amount=7 hash=0x2d957be559e54ec232fc02ad0129b9f8f751bc63fe41884b4183d190dec2a80b header sending=29.718µs log sending=823ns
may 31 14:54:25 root-pc erigon[25470]: [INFO] [05-31|14:54:25.685] head updated                             hash=0x2d957be559e54ec232fc02ad0129b9f8f751bc63fe41884b4183d190dec2a80b number=19989646
may 31 14:54:33 root-pc erigon[25470]: [INFO] [05-31|14:54:33.843] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
may 31 14:54:34 root-pc erigon[25470]: [INFO] [05-31|14:54:34.453] RPC Daemon notified of new headers       from=19989646 to=19989647 amount=1 hash=0x1ef4f00d4ff68db2b3504d0189686ce17d3f2bf1aaeb0aff69b6266885d03d89 header sending=20.97µs log sending=435ns
may 31 14:54:34 root-pc erigon[25470]: [INFO] [05-31|14:54:34.453] head updated                             hash=0x1ef4f00d4ff68db2b3504d0189686ce17d3f2bf1aaeb0aff69b6266885d03d89 number=19989647
may 31 14:54:40 root-pc erigon[25470]: [INFO] [05-31|14:54:40.484] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
may 31 14:54:40 root-pc erigon[25470]: [INFO] [05-31|14:54:40.870] RPC Daemon notified of new headers       from=19989647 to=19989648 amount=1 hash=0x888030af8c7e4e6f298446b47653749eee44949c34fd68524eccf830c5d56bd0 header sending=17.908µs log sending=769ns
may 31 14:54:40 root-pc erigon[25470]: [INFO] [05-31|14:54:40.870] head updated                             hash=0x888030af8c7e4e6f298446b47653749eee44949c34fd68524eccf830c5d56bd0 number=19989648
may 31 14:54:54 root-pc erigon[25470]: [INFO] [05-31|14:54:54.429] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
may 31 14:54:54 root-pc erigon[25470]: [INFO] [05-31|14:54:54.987] RPC Daemon notified of new headers       from=19989648 to=19989649 amount=1 hash=0xf42ddf15f925ae45ec8a027dd5e22015c8ccf818b66e81bef8c7e3599fe606dc header sending=36.065µs log sending=403ns
may 31 14:54:54 root-pc erigon[25470]: [INFO] [05-31|14:54:54.987] head updated                             hash=0xf42ddf15f925ae45ec8a027dd5e22015c8ccf818b66e81bef8c7e3599fe606dc number=19989649
may 31 14:55:10 root-pc erigon[25470]: [INFO] [05-31|14:55:10.164] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
may 31 14:55:10 root-pc erigon[25470]: [INFO] [05-31|14:55:10.625] RPC Daemon notified of new headers       from=19989649 to=19989650 amount=1 hash=0xbfc59b709e0f9f7d03bc6df62f390e3dd3378ebecbf29ddd9e88b470c5ff6981 header sending=42.148µs log sending=415ns
may 31 14:55:10 root-pc erigon[25470]: [INFO] [05-31|14:55:10.625] head updated                             hash=0xbfc59b709e0f9f7d03bc6df62f390e3dd3378ebecbf29ddd9e88b470c5ff6981 number=19989650
may 31 14:55:11 root-pc erigon[25470]: [INFO] [05-31|14:55:11.684] P2P                                      app=caplin peers=65
may 31 14:55:22 root-pc erigon[25470]: [INFO] [05-31|14:55:22.605] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
may 31 14:55:23 root-pc erigon[25470]: [INFO] [05-31|14:55:23.232] RPC Daemon notified of new headers       from=19989650 to=19989651 amount=1 hash=0xf11d1dba8ad6c4824bc04f9f391b80a269918e273b139784401faa8a1dfe18fd header sending=23.27µs log sending=375ns
may 31 14:55:23 root-pc erigon[25470]: [INFO] [05-31|14:55:23.232] head updated                             hash=0xf11d1dba8ad6c4824bc04f9f391b80a269918e273b139784401faa8a1dfe18fd number=19989651
may 31 14:56:07 root-pc erigon[25470]: [INFO] [05-31|14:56:07.588] [4/12 Execution] Completed on            block=19989655
may 31 14:56:09 root-pc erigon[25470]: [INFO] [05-31|14:56:09.917] RPC Daemon notified of new headers       from=19989651 to=19989655 amount=4 hash=0x6648411f7ebce6228c79136c8324ac65178e22cdb79787ae5dab99a8a431526e header sending=48.573µs log sending=537ns
may 31 14:56:09 root-pc erigon[25470]: [INFO] [05-31|14:56:09.918] head updated                             hash=0x6648411f7ebce6228c79136c8324ac65178e22cdb79787ae5dab99a8a431526e number=19989655
may 31 14:56:11 root-pc erigon[25470]: [INFO] [05-31|14:56:11.684] P2P                                      app=caplin peers=65
may 31 14:56:17 root-pc erigon[25470]: [INFO] [05-31|14:56:17.573] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
may 31 14:56:18 root-pc erigon[25470]: [INFO] [05-31|14:56:18.052] RPC Daemon notified of new headers       from=19989655 to=19989656 amount=1 hash=0x7b8e8bd842d59eeb5fc14644b095a8bf9d6625b0ee354e98638609230df26175 header sending=18.868µs log sending=648ns
may 31 14:56:18 root-pc erigon[25470]: [INFO] [05-31|14:56:18.053] head updated                             hash=0x7b8e8bd842d59eeb5fc14644b095a8bf9d6625b0ee354e98638609230df26175 number=19989656
may 31 14:56:31 root-pc erigon[25470]: [INFO] [05-31|14:56:31.045] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
may 31 14:56:31 root-pc erigon[25470]: [INFO] [05-31|14:56:31.910] RPC Daemon notified of new headers       from=19989656 to=19989657 amount=1 hash=0x1d8a8bc82a3daf47dc4acc2ba888578e01550e3133fe45e63cc4fd78dfea7ab5 header sending=17.523µs log sending=16.9µs
may 31 14:56:31 root-pc erigon[25470]: [INFO] [05-31|14:56:31.910] head updated                             hash=0x1d8a8bc82a3daf47dc4acc2ba888578e01550e3133fe45e63cc4fd78dfea7ab5 number=19989657
may 31 14:56:33 root-pc erigon[25470]: [INFO] [05-31|14:56:33.875] [p2p] GoodPeers                          eth68=33 eth67=7
may 31 14:56:34 root-pc erigon[25470]: [INFO] [05-31|14:56:34.478] [txpool] stat                            pending=102 baseFee=2761 queued=483 alloc=8.7GB sys=18.4GB
may 31 14:56:35 root-pc erigon[25470]: [INFO] [05-31|14:56:35.282] [mem] memory stats                       Rss=21.6MB Size=0B Pss=21.6MB SharedClean=2.7KB SharedDirty=0B PrivateClean=3.2MB PrivateDirty=18.4MB Referenced=21.4MB Anonymous=18.4MB Swap=104.2KB alloc=8.7GB sys=18.4GB
may 31 14:56:44 root-pc erigon[25470]: [INFO] [05-31|14:56:44.689] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
may 31 14:56:45 root-pc erigon[25470]: [INFO] [05-31|14:56:45.105] RPC Daemon notified of new headers       from=19989657 to=19989658 amount=1 hash=0xf72d6f41bb4688ebc675b6c45394057d6919011a5326e4864f539b3482726d3f header sending=64.941µs log sending=784ns
may 31 14:56:45 root-pc erigon[25470]: [INFO] [05-31|14:56:45.106] head updated                             hash=0xf72d6f41bb4688ebc675b6c45394057d6919011a5326e4864f539b3482726d3f number=19989658
may 31 14:56:58 root-pc erigon[25470]: [INFO] [05-31|14:56:58.875] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
may 31 14:56:59 root-pc erigon[25470]: [INFO] [05-31|14:56:59.577] RPC Daemon notified of new headers       from=19989658 to=19989659 amount=1 hash=0xc36f7c055b741dcc041178b9f614fbdcf143e6bc704f2f2f9f63ea0a11079be1 header sending=28.717µs log sending=390ns
may 31 14:56:59 root-pc erigon[25470]: [INFO] [05-31|14:56:59.577] head updated                             hash=0xc36f7c055b741dcc041178b9f614fbdcf143e6bc704f2f2f9f63ea0a11079be1 number=19989659
may 31 14:57:05 root-pc erigon[25470]: [INFO] [05-31|14:57:05.602] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
may 31 14:57:06 root-pc erigon[25470]: [INFO] [05-31|14:57:06.080] RPC Daemon notified of new headers       from=19989659 to=19989660 amount=1 hash=0xce5f6534d645f9af2c38736c1114ff0bc7bb9b25b3d31eb2f80a64fed40ab335 header sending=21.575µs log sending=432ns
may 31 14:57:06 root-pc erigon[25470]: [INFO] [05-31|14:57:06.080] head updated                             hash=0xce5f6534d645f9af2c38736c1114ff0bc7bb9b25b3d31eb2f80a64fed40ab335 number=19989660
may 31 14:57:11 root-pc erigon[25470]: [INFO] [05-31|14:57:11.687] P2P                                      app=caplin peers=64
may 31 14:57:21 root-pc erigon[25470]: [INFO] [05-31|14:57:21.131] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
may 31 14:57:21 root-pc erigon[25470]: [INFO] [05-31|14:57:21.407] RPC Daemon notified of new headers       from=19989660 to=19989661 amount=1 hash=0x40311ace1f2187ad1ab412fd9d0c0df21804566ff5791d11229cf32634a6a370 header sending=18.236µs log sending=440ns
may 31 14:57:21 root-pc erigon[25470]: [INFO] [05-31|14:57:21.407] head updated                             hash=0x40311ace1f2187ad1ab412fd9d0c0df21804566ff5791d11229cf32634a6a370 number=19989661
may 31 14:57:41 root-pc erigon[25470]: [INFO] [05-31|14:57:41.998] [4/12 Execution] Completed on            block=19989663
may 31 14:57:43 root-pc erigon[25470]: [INFO] [05-31|14:57:43.149] RPC Daemon notified of new headers       from=19989661 to=19989663 amount=2 hash=0x945111615faa2c36c4d8e42c11251d3cb12fc8ead0824f16d73273d82cf6c356 header sending=29.016µs log sending=874ns
may 31 14:57:43 root-pc erigon[25470]: [INFO] [05-31|14:57:43.149] head updated                             hash=0x945111615faa2c36c4d8e42c11251d3cb12fc8ead0824f16d73273d82cf6c356 number=19989663
may 31 14:57:56 root-pc erigon[25470]: [INFO] [05-31|14:57:56.472] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
may 31 14:57:57 root-pc erigon[25470]: [INFO] [05-31|14:57:57.173] RPC Daemon notified of new headers       from=19989663 to=19989664 amount=1 hash=0xc167c86c16a4493db019dba6e42eb008f14398c9e036e61ea711fec072889bd5 header sending=23.906µs log sending=813ns
may 31 14:57:57 root-pc erigon[25470]: [INFO] [05-31|14:57:57.173] head updated                             hash=0xc167c86c16a4493db019dba6e42eb008f14398c9e036e61ea711fec072889bd5 number=19989664
may 31 14:58:07 root-pc erigon[25470]: [INFO] [05-31|14:58:07.671] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
may 31 14:58:08 root-pc erigon[25470]: [INFO] [05-31|14:58:08.192] RPC Daemon notified of new headers       from=19989664 to=19989665 amount=1 hash=0x25e36271ad2208307bb1163c25c86f05a27647c94258495af90bf3691978b25e header sending=14.559µs log sending=395ns
may 31 14:58:08 root-pc erigon[25470]: [INFO] [05-31|14:58:08.192] head updated                             hash=0x25e36271ad2208307bb1163c25c86f05a27647c94258495af90bf3691978b25e number=19989665
may 31 14:58:11 root-pc erigon[25470]: [INFO] [05-31|14:58:11.684] P2P                                      app=caplin peers=65
may 31 14:58:22 root-pc erigon[25470]: [INFO] [05-31|14:58:22.336] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
may 31 14:58:22 root-pc erigon[25470]: [INFO] [05-31|14:58:22.885] RPC Daemon notified of new headers       from=19989665 to=19989666 amount=1 hash=0xfe054ead572bc840467a5eadf987d48d6812e80c6ff6c30f8937fd529cf6e381 header sending=17.048µs log sending=404ns
may 31 14:58:22 root-pc erigon[25470]: [INFO] [05-31|14:58:22.885] head updated                             hash=0xfe054ead572bc840467a5eadf987d48d6812e80c6ff6c30f8937fd529cf6e381 number=19989666
may 31 14:58:31 root-pc erigon[25470]: [INFO] [05-31|14:58:31.206] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
may 31 14:58:31 root-pc erigon[25470]: [INFO] [05-31|14:58:31.625] RPC Daemon notified of new headers       from=19989666 to=19989667 amount=1 hash=0xbc0eb5a64e95863d968631fd0c67876a22a142077803384b2f8c91567859cb3e header sending=21.971µs log sending=421ns
may 31 14:58:31 root-pc erigon[25470]: [INFO] [05-31|14:58:31.625] head updated                             hash=0xbc0eb5a64e95863d968631fd0c67876a22a142077803384b2f8c91567859cb3e number=19989667
may 31 14:59:11 root-pc erigon[25470]: [INFO] [05-31|14:59:11.039] [4/12 Execution] Completed on            block=19989670
may 31 14:59:11 root-pc erigon[25470]: [INFO] [05-31|14:59:11.684] P2P                                      app=caplin peers=63
may 31 14:59:12 root-pc erigon[25470]: [INFO] [05-31|14:59:12.673] RPC Daemon notified of new headers       from=19989667 to=19989670 amount=3 hash=0x25e72f9bd7613315bd701d2725900e58e4a2612b9877a1580478680d6015d5fa header sending=23.577µs log sending=423ns
may 31 14:59:12 root-pc erigon[25470]: [INFO] [05-31|14:59:12.674] head updated                             hash=0x25e72f9bd7613315bd701d2725900e58e4a2612b9877a1580478680d6015d5fa number=19989670
may 31 14:59:33 root-pc erigon[25470]: [INFO] [05-31|14:59:33.875] [p2p] GoodPeers                          eth67=7 eth68=33
may 31 14:59:34 root-pc erigon[25470]: [INFO] [05-31|14:59:34.475] [txpool] stat                            pending=423 baseFee=2738 queued=475 alloc=9.1GB sys=18.4GB
may 31 14:59:35 root-pc erigon[25470]: [INFO] [05-31|14:59:35.364] [mem] memory stats                       Rss=22.5MB Size=0B Pss=22.5MB SharedClean=2.7KB SharedDirty=0B PrivateClean=3.3MB PrivateDirty=19.2MB Referenced=22.3MB Anonymous=19.2MB Swap=104.2KB alloc=9.2GB sys=18.4GB
may 31 15:00:10 root-pc erigon[25470]: [INFO] [05-31|15:00:10.324] [4/12 Execution] Completed on            block=19989675
may 31 15:00:11 root-pc erigon[25470]: [INFO] [05-31|15:00:11.684] P2P                                      app=caplin peers=62
may 31 15:00:12 root-pc erigon[25470]: [INFO] [05-31|15:00:12.474] RPC Daemon notified of new headers       from=19989670 to=19989675 amount=5 hash=0xa526cd419a13e841f0329d5fa31edc6c027b6c85c1a1a5ecd8e9242313253134 header sending=36.072µs log sending=455ns
may 31 15:00:12 root-pc erigon[25470]: [INFO] [05-31|15:00:12.474] head updated                             hash=0xa526cd419a13e841f0329d5fa31edc6c027b6c85c1a1a5ecd8e9242313253134 number=19989675
may 31 15:01:11 root-pc erigon[25470]: [INFO] [05-31|15:01:11.684] P2P                                      app=caplin peers=57
may 31 15:01:59 root-pc erigon[25470]: [INFO] [05-31|15:01:59.572] [4/12 Execution] Completed on            block=19989684
may 31 15:02:04 root-pc erigon[25470]: [INFO] [05-31|15:02:04.198] RPC Daemon notified of new headers       from=19989675 to=19989684 amount=9 hash=0x8964cd8904e6757fb850cda8a00bc6973154b3bed9025cbe309759fb203b1713 header sending=43.036µs log sending=1.037µs
may 31 15:02:04 root-pc erigon[25470]: [INFO] [05-31|15:02:04.198] head updated                             hash=0x8964cd8904e6757fb850cda8a00bc6973154b3bed9025cbe309759fb203b1713 number=19989684
may 31 15:02:11 root-pc erigon[25470]: [INFO] [05-31|15:02:11.683] P2P                                      app=caplin peers=63
may 31 15:02:33 root-pc erigon[25470]: [INFO] [05-31|15:02:33.876] [p2p] GoodPeers                          eth68=33 eth67=7
may 31 15:02:34 root-pc erigon[25470]: [INFO] [05-31|15:02:34.477] [txpool] stat                            pending=467 baseFee=2916 queued=475 alloc=12.4GB sys=18.4GB
may 31 15:02:35 root-pc erigon[25470]: [INFO] [05-31|15:02:35.274] [mem] memory stats                       Rss=22.2MB Size=0B Pss=22.2MB SharedClean=2.7KB SharedDirty=0B PrivateClean=4.5MB PrivateDirty=17.7MB Referenced=21.6MB Anonymous=17.7MB Swap=104.2KB alloc=12.4GB sys=18.4GB
may 31 15:03:11 root-pc erigon[25470]: [INFO] [05-31|15:03:11.684] P2P                                      app=caplin peers=64
may 31 15:04:11 root-pc erigon[25470]: [INFO] [05-31|15:04:11.684] P2P                                      app=caplin peers=60
may 31 15:05:11 root-pc erigon[25470]: [INFO] [05-31|15:05:11.684] P2P                                      app=caplin peers=60
may 31 15:05:33 root-pc erigon[25470]: [INFO] [05-31|15:05:33.876] [p2p] GoodPeers                          eth68=33 eth67=7
may 31 15:05:34 root-pc erigon[25470]: [INFO] [05-31|15:05:34.477] [txpool] stat                            pending=2092 baseFee=3092 queued=561 alloc=9.0GB sys=18.4GB
may 31 15:05:35 root-pc erigon[25470]: [INFO] [05-31|15:05:35.246] [mem] memory stats                       Rss=23.3MB Size=0B Pss=23.3MB SharedClean=2.7KB SharedDirty=0B PrivateClean=4.7MB PrivateDirty=18.6MB Referenced=22.4MB Anonymous=18.6MB Swap=104.2KB alloc=9.0GB sys=18.4GB
may 31 15:06:10 root-pc erigon[25470]: [INFO] [05-31|15:06:10.176] [3/12 Senders] Started                   from=19989684 to=19989704
may 31 15:06:10 root-pc erigon[25470]: [INFO] [05-31|15:06:10.213] [4/12 Execution] Blocks execution        from=19989684 to=19989704
may 31 15:06:11 root-pc erigon[25470]: [INFO] [05-31|15:06:11.684] P2P                                      app=caplin peers=60
may 31 15:06:11 root-pc erigon[25470]: [INFO] [05-31|15:06:11.726] [4/12 Execution] Completed on            block=19989704
may 31 15:06:11 root-pc erigon[25470]: [INFO] [05-31|15:06:11.726] [5/12 HashState] Promoting plain state   from=19989684 to=19989704
may 31 15:06:11 root-pc erigon[25470]: [INFO] [05-31|15:06:11.726] [5/12 HashState] Incremental promotion   from=19989684 to=19989704 codes=true csbucket=AccountChangeSet
may 31 15:06:11 root-pc erigon[25470]: [INFO] [05-31|15:06:11.915] [5/12 HashState] Incremental promotion   from=19989684 to=19989704 codes=false csbucket=AccountChangeSet
may 31 15:06:12 root-pc erigon[25470]: [INFO] [05-31|15:06:12.045] [5/12 HashState] Incremental promotion   from=19989684 to=19989704 codes=false csbucket=StorageChangeSet
may 31 15:06:12 root-pc erigon[25470]: [INFO] [05-31|15:06:12.375] [6/12 IntermediateHashes] Generating intermediate hashes from=19989684 to=19989704
may 31 15:06:13 root-pc erigon[25470]: [INFO] [05-31|15:06:13.593] [7/12 CallTraces] Pruned call trace intermediate table from=19899684 to=19899703
may 31 15:06:18 root-pc erigon[25470]: [INFO] [05-31|15:06:18.437] RPC Daemon notified of new headers       from=19989684 to=19989704 amount=20 hash=0xe9805c425ba6fac2adff74ae2f7dcc3e37305368d3251ceb2ae45d978650013a header sending=96.021µs log sending=380ns

How I installed Erigon: everything by instructions https://erigon.gitbook.io/erigon/eli5-nodes/how-to-run-an-ethereum-node

fridary commented 4 months ago

@AskAlexSharov hi, did you check? I installed Erigon from the box, everything with default settings and high performed server (proof is Geth fast syncing).

AskAlexSharov commented 4 months ago

metric db_commit_seconds looks good

metric block_consumer_delay looks bad:

block_consumer_delay{type="header_download",quantile="0.5"} 8.469509384
block_consumer_delay{type="header_download",quantile="0.9"} 18.172671189
block_consumer_delay{type="header_download",quantile="0.97"} 21.707992357

so, i guess problem is somewhere before erigon got header. @fridary Show du -hsc datadir/*

AskAlexSharov commented 4 months ago

anyway also show pls:

go tool pprof -inuse_space -png http://127.0.0.1:6060/debug/pprof/heap > mem.png
go tool pprof -png  http://127.0.0.1:6060/debug/pprof/profile\?seconds\=120 > cpu.png

fridary commented 4 months ago

@AskAlexSharov the main reason why I write is that Geth works good (2 seconds delay instead 2-50 seconds on Erigon).

$ du -hsc /disk_sde/erigon/*
0   /disk_sde/erigon/LOCK
106G    /disk_sde/erigon/caplin
1,4T    /disk_sde/erigon/chaindata
23M /disk_sde/erigon/downloader
319M    /disk_sde/erigon/logs
4,0K    /disk_sde/erigon/nodekey
31M /disk_sde/erigon/nodes
677G    /disk_sde/erigon/snapshots
4,1G    /disk_sde/erigon/temp
1,1G    /disk_sde/erigon/txpool
2,2T    total

mem

cpu

AskAlexSharov commented 4 months ago

@fridary my advise:

rm -rf /disk_sde/erigon/caplin
rm -rf /disk_sde/erigon/txpool
rm -rf /disk_sde/erigon/temp
add --no-downloader --snap.stop
then check if it helped

fridary commented 4 months ago

@AskAlexSharov I do did --no-downloader --snap.stop 1 week ago as you suggested me. rm I have deleted, waited 1 day and got results range 5-12 seconds delay by getting newHeads with sockets, although Geth returned 1.5-3 seconds delay. Anything more we can do?

AskAlexSharov commented 4 months ago

@Giulio2002 do you know what metric can take a look in Caplin to understand header delay: https://github.com/ledgerwatch/erigon/issues/10563#issuecomment-2151337062 ?

erigontech / erigon

newHeads eth_subscribe has 2-15 seconds delay although Geth only 2-3 seconds #10563