node can not synced

[sampleblood]

description: start node from 0926-snapshot and not synced after a day.

free -m:

free -m
               total        used        free      shared  buff/cache   available
Mem:           31801        5931         262           6       25608       25438
Swap:              0           0           0

log:

lots of this log:
info  2024-09-28T11:10:29.398 net-1     net_plugin.cpp:4863           connection_monitor   ] p2p client connections: 0/25, peer connections: 9/9, block producer peers: 0
info  2024-09-28T11:10:29.723 net-3     net_plugin.cpp:2308           recv_handshake       ] ["eosn-eos-seed172:9876 - 9876868" - 1 209.249.216.152:9876] handshake lib 396586893, fhead 396586895, id d4e8586275a8521f.. sync 1, fhead 396400117, lib 396399934
info  2024-09-28T11:10:29.724 net-3     net_plugin.cpp:1604           operator()           ] ["eosn-eos-seed172:9876 - 9876868" - 1 209.249.216.152:9876] Sending handshake generation 1559, lib 396399934, fhead 396400117, id 02113d6d03fc2f9a
info  2024-09-28T11:10:32.792 net-1     net_plugin.cpp:2308           recv_handshake       ] ["61.35.89.227:30333 - b884d1d" - 7 61.35.89.223:30333] handshake lib 396586899, fhead 396586901, id fb00a7a7e6e54d78.. sync 1, fhead 396400117, lib 396399934
info  2024-09-28T11:10:32.792 net-1     net_plugin.cpp:1604           operator()           ] ["61.35.89.227:30333 - b884d1d" - 7 61.35.89.223:30333] Sending handshake generation 782, lib 396399934, fhead 396400117, id 02113d6d03fc2f9a
info  2024-09-28T11:10:33.411 net-0     net_plugin.cpp:2308           recv_handshake       ] ["p2p.eosflare.io:9876 - e0f1531" - 5 173.231.59.114:9876] handshake lib 396586901, fhead 396586903, id e08854f5fd9863cd.. sync 1, fhead 396400117, lib 396399934

few this log:
Received block a1a926745b47fd5f... #396400000 @ 2024-09-27T08:08:43.000 signed by atticlabeosb [trxs: 13, lib: 396399934, net: 1856, cpu: 3288, elapsed: 7425, time: 16037, latency: 97004816 ms]

software: nodeos version v1.0.1 v1.0.1-574650744460373f635d48cac9aa6dee67dcbfdb

system: aws r5a.xlarge with 4 vCPUs, 32.0 GiB of memory, 200GB ssd

config.ini:

abi-serializer-max-time-ms = 2000
chain-state-db-size-mb = 131072
enable-account-queries = true
http-max-response-time-ms = 300
verbose-http-errors = true
http-threads = 8
# Maximum number of client nodes from any single IP address (eosio::net_plugin)
p2p-max-nodes-per-host = 2

# alohaeosprod: GB, London
p2p-peer-address = peer.main.alohaeos.com:9876

# aus1genereos: AU, Sydney
p2p-peer-address = p2p.genereos.io:9876

# bp4dex223dao: US, United States
p2p-peer-address = p2p.dex223.io:9876

# cryptolions1: DE, Germany-Finland
p2p-peer-address = p2p.eos.cryptolions.io:9876

# eosamsterdam: DE, eos01
p2p-peer-address = mainnet.eosamsterdam.net:9876

# eoseouldotio: KR, Seoul
p2p-peer-address = p2p2.eoseoul.io:30333

# eosflareiobp: US, Ogden, Utah
p2p-peer-address = p2p.eosflare.io:9876

# eosnationftw: CA, Canada
p2p-peer-address = eos.seed.eosnation.io:9876

# eosphereiobp: CA, Beauharnois
p2p-peer-address = peer1.eosphere.io:9876

net-threads = 5

plugin = eosio::chain_plugin
plugin = eosio::chain_api_plugin
plugin = eosio::net_api_plugin
plugin = eosio::db_size_api_plugin

[heifner]

For aws, we would recommend something more like z1d.6xlarge.

[sampleblood]

z1d.6xlarge

just used as an api node, not a block producer, r5a.large was enough before spring upgrade,spring api node must use z1d.6xlarge ?

[sampleblood]

z1d.6xlarge

just used as an api node, not a block producer, r5a.large was enough before spring upgrade,spring api node must use z1d.6xlarge ?

aws r5a.2xlarge (64G RAM and some swap) catch up for an api node.

[the-smooth-operator]

we are having the same issue. Nodes are struggling to process blocks, RPC API stops responding. Snapshots take a long time to process and, once processed, the node doesn't start syncing up blocks.

We have noticed a huge increase in data per block on Friday and again today.

What is the recommended configuration for running a node that can keep up? Looking for advice on modifications to config.ini like number of net/http threads etc...

[bhazzard]

z1d.6xlarge

just used as an api node, not a block producer, r5a.large was enough before spring upgrade,spring api node must use z1d.6xlarge ?

It isn't Spring that requires more memory. Recent usage on the EOS network has written a lot of data to RAM, causing the state database to multiply in size.

So now that the state database is much larger, these larger instances are required.