search

erigontech / erigon

Ethereum implementation on the efficiency frontier https://erigon.gitbook.io
GNU Lesser General Public License v3.0
3.12k stars 1.11k forks source link

The polygon network cannot obtain new block data. #8636

Open presIey opened 11 months ago

presIey commented 11 months ago

System information

erigon version 2.52.5-7c5e4e77

OS & Version: NAME="Rocky Linux" VERSION="9.2 (Blue Onyx)"

Commit hash:

Erigon Command (with flags/config): cat start_erigon_polygon.sh /home/one/erigon/build/bin/erigon \ --datadir=/mnt/data_lvmu2/polygon/erigon \ --chain=bor-mainnet \ --snapshots=true \ --bor.heimdall=http://localhost:1317 \ --bor.heimdallgRPC=http://localhost:3132 \ --torrent.download.rate="500mb" \ --txpool.api.addr=14006 \ --port=30304 \ --http \ --http.addr="0.0.0.0" \ --http.port=14005 \ --torrent.port=14008 \ --http.api=eth,erigon,engine,web3,net,txpool \ --private.api.addr="127.0.0.1:9195" \ --http.corsdomain '*' \ --ws Consensus Layer: heimdall Consensus Layer Command (with flags/config):

Chain/Network: polygon mainnet

Expected behaviour

Automatically sync new block data.

Actual behaviour

Unable to get new block data

Steps to reproduce the behaviour

Normal startup script

Backtrace


Nov 01 17:06:42 105 systemd[1]: Started erigon polygon Client.
Nov 01 17:06:43 105 bash[2271956]: [INFO] [11-01|17:06:43.291] logging to file system                   log dir=/mnt/data_lvmu2/polygon/erigon/logs file prefix=erigon log level=info json=false
Nov 01 17:06:43 105 bash[2271956]: [INFO] [11-01|17:06:43.291] Build info                               git_branch=HEAD git_tag=v2.52.5 git_commit=7c5e4e770f122299c34a9309a37a33a146443da8
Nov 01 17:06:43 105 bash[2271956]: [INFO] [11-01|17:06:43.291] Starting Erigon on Bor Mainnet...
Nov 01 17:06:43 105 bash[2271956]: [INFO] [11-01|17:06:43.293] Maximum peer count                       ETH=100 total=100
Nov 01 17:06:43 105 bash[2271956]: [INFO] [11-01|17:06:43.293] starting HTTP APIs                       APIs=eth,erigon,engine,web3,net,txpool
Nov 01 17:06:43 105 bash[2271956]: [INFO] [11-01|17:06:43.293] torrent verbosity                        level=WRN
Nov 01 17:06:48 105 bash[2271956]: [INFO] [11-01|17:06:48.398] Set global gas cap                       cap=50000000
Nov 01 17:06:48 105 bash[2271956]: [INFO] [11-01|17:06:48.400] [Downloader] Runnning with               ipv6-enabled=true ipv4-enabled=true download.rate=500mb upload.rate=4mb
Nov 01 17:06:48 105 bash[2271956]: [INFO] [11-01|17:06:48.400] Opening Database                         label=chaindata path=/mnt/data_lvmu2/polygon/erigon/chaindata
Nov 01 17:06:48 105 bash[2271956]: [INFO] [11-01|17:06:48.402] Initialised chain configuration          config="{ChainID: 137, Homestead: 0, DAO: <nil>, Tangerine Whistle: 0, Spurious Dragon: 0, Byzantium: 0, Constantinople: 0, Petersburg: 0, Istanbul: 3395000, Muir Glacier: 3395000, Berlin: 14750000, London: 23850000, Arrow Glacier: <nil>, Gray Glacier: <nil>, Terminal Total Difficulty: <nil>, Merge Netsplit: <nil>, Shanghai: <nil>, Cancun: <nil>, Prague: <nil>, Engine: bor}" genesis=0xa9c28ce2141b56c474f1dc504bee9b01eb1bd7d1a507580d5519d4437a97de1b
Nov 01 17:06:48 105 bash[2271956]: [INFO] [11-01|17:06:48.406] Initialising Ethereum protocol           network=137
Nov 01 17:06:48 105 bash[2271956]: [INFO] [11-01|17:06:48.407] Connected to Heimdall gRPC server        address=http://localhost:3132
Nov 01 17:06:48 105 bash[2271956]: [INFO] [11-01|17:06:48.407] Opening Database                         label=bor path=/mnt/data_lvmu2/polygon/erigon/bor
Nov 01 17:06:49 105 bash[2271956]: [INFO] [11-01|17:06:49.791] Starting private RPC server              on=127.0.0.1:9195
Nov 01 17:06:49 105 bash[2271956]: [INFO] [11-01|17:06:49.791] new subscription to logs established
Nov 01 17:06:49 105 bash[2271956]: [INFO] [11-01|17:06:49.792] rpc filters: subscribing to Erigon events
Nov 01 17:06:49 105 bash[2271956]: [INFO] [11-01|17:06:49.792] New txs subscriber joined
Nov 01 17:06:49 105 bash[2271956]: [INFO] [11-01|17:06:49.792] new subscription to newHeaders established
Nov 01 17:06:49 105 bash[2271956]: [INFO] [11-01|17:06:49.792] Reading JWT secret                       path=/mnt/data_lvmu2/polygon/erigon/jwt.hex
Nov 01 17:06:49 105 bash[2271956]: [INFO] [11-01|17:06:49.793] HTTP endpoint opened                     url=[::]:14005 ws=true ws.compression=true grpc=false
Nov 01 17:06:49 105 bash[2271956]: [INFO] [11-01|17:06:49.793] HTTP endpoint opened for Engine API      url=127.0.0.1:8551 ws=true ws.compression=true
Nov 01 17:06:49 105 bash[2271956]: [WARN] [11-01|17:06:49.794] NAT ExternalIP resolution has failed, try to pass a different --nat option err="no UPnP or NAT-PMP router discovered"
Nov 01 17:06:49 105 bash[2271956]: [WARN] [11-01|17:06:49.796] NAT ExternalIP resolution has failed, try to pass a different --nat option err="no UPnP or NAT-PMP router discovered"
Nov 01 17:06:49 105 bash[2271956]: [INFO] [11-01|17:06:49.796] Started P2P networking                   version=68 self=enode://2d03a141a6a25ba5a04e767de1f3d889b30eb3cdeb977d9155c50da616348b4b0abd339927fcd512ee62ccb673dbaa45d9982b8e1a3fb986e03175032080aa90@127.0.0.1:0 name=erigon/v2.52.5-7c5e4e77/linux-amd64/go1.21.3
Nov 01 17:06:49 105 bash[2271956]: [INFO] [11-01|17:06:49.797] Started P2P networking                   version=67 self=enode://2d03a141a6a25ba5a04e767de1f3d889b30eb3cdeb977d9155c50da616348b4b0abd339927fcd512ee62ccb673dbaa45d9982b8e1a3fb986e03175032080aa90@127.0.0.1:0 name=erigon/v2.52.5-7c5e4e77/linux-amd64/go1.21.3
Nov 01 17:06:49 105 bash[2271956]: [INFO] [11-01|17:06:49.802] Fetching checkpoint                      number=-1
Nov 01 17:06:49 105 bash[2271956]: [INFO] [11-01|17:06:49.802] Fetching milestone
Nov 01 17:06:49 105 bash[2271956]: [INFO] [11-01|17:06:49.802] Fetching latest no ack milestone Id
Nov 01 17:06:49 105 bash[2271956]: [INFO] [11-01|17:06:49.802] [2/15 Headers] Waiting for headers...    from=47278798
Nov 01 17:06:52 105 bash[2271956]: [INFO] [11-01|17:06:52.203] [txpool] Started
Nov 01 17:06:52 105 bash[2271956]: [INFO] [11-01|17:06:52.787] Span with given block number is not loaded fetching span=7388
Nov 01 17:06:52 105 bash[2271956]: [INFO] [11-01|17:06:52.787] Fetching span                            spanID=7388
Nov 01 17:06:53 105 bash[2271956]: [EROR] [11-01|17:06:53.803] Failed to fetch latest no-ack milestone  err="rpc error: code = DeadlineExceeded desc = context deadline exceeded"
Nov 01 17:06:53 105 bash[2271956]: [WARN] [11-01|17:06:53.803] unable to start the no-ack-milestone service service - first run err="failed to fetch latest milestone"
Nov 01 17:06:59 105 bash[2271956]: [INFO] [11-01|17:06:59.804] Fetching latest no ack milestone Id
Nov 01 17:07:03 105 bash[2271956]: [EROR] [11-01|17:07:03.805] Failed to fetch latest no-ack milestone  err="rpc error: code = DeadlineExceeded desc = context deadline exceeded"
Nov 01 17:07:03 105 bash[2271956]: [WARN] [11-01|17:07:03.805] unable to handle no-ack-milestone service err="failed to fetch latest milestone"
Nov 01 17:07:05 105 bash[2271956]: [INFO] [11-01|17:07:05.803] Fetching latest no ack milestone Id
Nov 01 17:07:09 105 bash[2271956]: [EROR] [11-01|17:07:09.804] Failed to fetch latest no-ack milestone  err="rpc error: code = DeadlineExceeded desc = context deadline exceeded"
Nov 01 17:07:09 105 bash[2271956]: [WARN] [11-01|17:07:09.804] unable to handle no-ack-milestone service err="failed to fetch latest milestone"
Nov 01 17:07:11 105 bash[2271956]: [INFO] [11-01|17:07:11.804] Fetching latest no ack milestone Id
Nov 01 17:07:15 105 bash[2271956]: [EROR] [11-01|17:07:15.805] Failed to fetch latest no-ack milestone  err="rpc error: code = DeadlineExceeded desc = context deadline exceeded"
Nov 01 17:07:15 105 bash[2271956]: [WARN] [11-01|17:07:15.805] unable to handle no-ack-milestone service err="failed to fetch latest milestone"
Nov 01 17:07:17 105 bash[2271956]: [INFO] [11-01|17:07:17.804] Fetching latest no ack milestone Id
Nov 01 17:07:19 105 bash[2271956]: [WARN] [11-01|17:07:19.802] unable to start the whitelist checkpoint service - first run err="failed to fetch latest checkpoint"
Nov 01 17:07:19 105 bash[2271956]: [EROR] [11-01|17:07:19.802] Failed to fetch latest milestone for whitelisting err="rpc error: code = DeadlineExceeded desc = context deadline exceeded"
Nov 01 17:07:19 105 bash[2271956]: [WARN] [11-01|17:07:19.803] unable to start the whitelist milestone service - first run err="failed to fetch latest milestone"
Nov 01 17:07:21 105 bash[2271956]: [EROR] [11-01|17:07:21.804] Failed to fetch latest no-ack milestone  err="rpc error: code = DeadlineExceeded desc = context deadline exceeded"
Nov 01 17:07:21 105 bash[2271956]: [WARN] [11-01|17:07:21.805] unable to handle no-ack-milestone service err="failed to fetch latest milestone"
Nov 01 17:07:23 105 bash[2271956]: [INFO] [11-01|17:07:23.804] Fetching latest no ack milestone Id
Nov 01 17:07:27 105 bash[2271956]: [EROR] [11-01|17:07:27.804] Failed to fetch latest no-ack milestone  err="rpc error: code = DeadlineExceeded desc = context deadline exceeded"
Nov 01 17:07:27 105 bash[2271956]: [WARN] [11-01|17:07:27.804] unable to handle no-ack-milestone service err="failed to fetch latest milestone"
Nov 01 17:07:29 105 bash[2271956]: [INFO] [11-01|17:07:29.803] Fetching latest no ack milestone Id
Nov 01 17:07:31 105 bash[2271956]: [INFO] [11-01|17:07:31.803] Fetching milestone
Nov 01 17:07:33 105 bash[2271956]: [EROR] [11-01|17:07:33.804] Failed to fetch latest no-ack milestone  err="rpc error: code = DeadlineExceeded desc = context deadline exceeded"
Nov 01 17:07:33 105 bash[2271956]: [WARN] [11-01|17:07:33.804] unable to handle no-ack-milestone service err="failed to fetch latest milestone"
Nov 01 17:07:35 105 bash[2271956]: [INFO] [11-01|17:07:35.803] Fetching latest no ack milestone Id
Nov 01 17:07:39 105 bash[2271956]: [EROR] [11-01|17:07:39.803] Failed to fetch latest no-ack milestone  err="rpc error: code = DeadlineExceeded desc = context deadline exceeded"
Nov 01 17:07:39 105 bash[2271956]: [WARN] [11-01|17:07:39.803] unable to handle no-ack-milestone service err="failed to fetch latest milestone"
Nov 01 17:07:41 105 bash[2271956]: [INFO] [11-01|17:07:41.803] Fetching latest no ack milestone Id
Nov 01 17:07:45 105 bash[2271956]: [EROR] [11-01|17:07:45.803] Failed to fetch latest no-ack milestone  err="rpc error: code = DeadlineExceeded desc = context deadline exceeded"
Nov 01 17:07:45 105 bash[2271956]: [WARN] [11-01|17:07:45.804] unable to handle no-ack-milestone service err="failed to fetch latest milestone"
Nov 01 17:07:47 105 bash[2271956]: [INFO] [11-01|17:07:47.803] Fetching latest no ack milestone Id
Nov 01 17:07:51 105 bash[2271956]: [EROR] [11-01|17:07:51.804] Failed to fetch latest no-ack milestone  err="rpc error: code = DeadlineExceeded desc = context deadline exceeded"
Nov 01 17:07:51 105 bash[2271956]: [WARN] [11-01|17:07:51.804] unable to handle no-ack-milestone service err="failed to fetch latest milestone"
Nov 01 17:07:53 105 bash[2271956]: [INFO] [11-01|17:07:53.803] Fetching latest no ack milestone Id
Nov 01 17:07:57 105 bash[2271956]: [EROR] [11-01|17:07:57.804] Failed to fetch latest no-ack milestone  err="rpc error: code = DeadlineExceeded desc = context deadline exceeded"
Nov 01 17:07:57 105 bash[2271956]: [WARN] [11-01|17:07:57.804] unable to handle no-ack-milestone service err="failed to fetch latest milestone"
Nov 01 17:07:59 105 bash[2271956]: [INFO] [11-01|17:07:59.803] Fetching latest no ack milestone Id
Nov 01 17:08:01 105 bash[2271956]: [EROR] [11-01|17:08:01.804] Failed to fetch latest milestone for whitelisting err="rpc error: code = DeadlineExceeded desc = context deadline exceeded"
Nov 01 17:08:01 105 bash[2271956]: [WARN] [11-01|17:08:01.804] unable to handle whitelist milestone     err="failed to fetch latest milestone"
Nov 01 17:08:01 105 bash[2271956]: [INFO] [11-01|17:08:01.804] Fetching milestone
Nov 01 17:08:03 105 bash[2271956]: [EROR] [11-01|17:08:03.804] Failed to fetch latest no-ack milestone  err="rpc error: code = DeadlineExceeded desc = context deadline exceeded"
Nov 01 17:08:03 105 bash[2271956]: [WARN] [11-01|17:08:03.804] unable to handle no-ack-milestone service err="failed to fetch latest milestone"
Nov 01 17:08:05 105 bash[2271956]: [INFO] [11-01|17:08:05.803] Fetching latest no ack milestone Id
Nov 01 17:08:09 105 bash[2271956]: [EROR] [11-01|17:08:09.804] Failed to fetch latest no-ack milestone  err="rpc error: code = DeadlineExceeded desc = context deadline exceeded"
Nov 01 17:08:09 105 bash[2271956]: [WARN] [11-01|17:08:09.804] unable to handle no-ack-milestone service err="failed to fetch latest milestone"
Nov 01 17:08:11 105 bash[2271956]: [INFO] [11-01|17:08:11.803] Fetching latest no ack milestone Id
Nov 01 17:08:15 105 bash[2271956]: [EROR] [11-01|17:08:15.804] Failed to fetch latest no-ack milestone  err="rpc error: code = DeadlineExceeded desc = context deadline exceeded"
Nov 01 17:08:15 105 bash[2271956]: [WARN] [11-01|17:08:15.804] unable to handle no-ack-milestone service err="failed to fetch latest milestone"

I downloaded the archive block data officially provided by polygon, but the latest block data cannot be passed through the client. I use archive mode.

cshintov commented 11 months ago

Seems your heimdall is down! What does it logs say?

presIey commented 11 months ago

[one@105 ~]$ journalctl -fu heimdalld_rest Nov 01 17:28:40 105 bash[2274764]: ERROR[2023-11-01|17:28:40.880] Cannot start REST server. module=rest-server Error="close tcp [::]:1317: use of closed network connection" Nov 01 17:28:40 105 bash[2274764]: ERROR: close tcp [::]:1317: use of closed network connection Nov 01 17:28:40 105 systemd[1]: heimdalld_rest.service: Deactivated successfully. Nov 01 17:28:40 105 systemd[1]: Stopped Polygon Pos Light Client. Nov 01 17:28:40 105 systemd[1]: Started Polygon Pos Light Client. Nov 01 17:28:40 105 bash[2274794]: DEBUG[2023-11-01|17:28:40.943] Missing ETH RPC timeout or invalid value provided, falling back to default timeout=5s Nov 01 17:28:40 105 bash[2274794]: DEBUG[2023-11-01|17:28:40.943] Missing BOR RPC timeout or invalid value provided, falling back to default timeout=5s Nov 01 17:28:41 105 bash[2274794]: INFO [2023-11-01|17:28:41.036] Starting application REST service (chain-id: "137")... module=rest-server Nov 01 17:28:41 105 bash[2274794]: INFO [2023-11-01|17:28:41.036] GRPC Server started module=gRPC-server addr=0.0.0.0:3132 Nov 01 17:28:41 105 bash[2274794]: INFO [2023-11-01|17:28:41.036] Starting RPC HTTP server on [::]:1317 module=rest-server

[one@105 node_scripts]$ journalctl -fu heimdalld Nov 01 17:29:17 105 bash[2257154]: ERROR[2023-11-01|17:29:17.730] Stopping peer for error module=p2p peer="Peer{MConn{15.204.220.98:26656} 78ba1f1d4a758be3b5884062cbcfe4a7009de930 out}" err=EOF Nov 01 17:29:18 105 bash[2257154]: ERROR[2023-11-01|17:29:18.021] Connection failed @ recvRoutine (reading byte) module=p2p peer=69619af3aafe098d5a77ab7a82147e7bb8f90c8d@195.189.97.15:49656 conn=MConn{195.189.97.15:49656} err="read tcp 192.168.1.105:50662->195.189.97.15:49656: read: connection reset by peer" Nov 01 17:29:18 105 bash[2257154]: ERROR[2023-11-01|17:29:18.021] Stopping peer for error module=p2p peer="Peer{MConn{195.189.97.15:49656} 69619af3aafe098d5a77ab7a82147e7bb8f90c8d out}" err="read tcp 192.168.1.105:50662->195.189.97.15:49656: read: connection reset by peer" Nov 01 17:29:18 105 bash[2257154]: ERROR[2023-11-01|17:29:18.295] Connection failed @ recvRoutine (reading byte) module=p2p peer=e999edce9189cec832e7e035486bc5aa0ab89377@169.155.45.109:26656 conn=MConn{169.155.45.109:26656} err="read tcp 192.168.1.105:55826->169.155.45.109:26656: read: connection reset by peer" Nov 01 17:29:18 105 bash[2257154]: ERROR[2023-11-01|17:29:18.295] Stopping peer for error module=p2p peer="Peer{MConn{169.155.45.109:26656} e999edce9189cec832e7e035486bc5aa0ab89377 out}" err="read tcp 192.168.1.105:55826->169.155.45.109:26656: read: connection reset by peer" Nov 01 17:29:18 105 bash[2257154]: INFO [2023-11-01|17:29:18.566] Executed block module=state height=16196200 validTxs=0 invalidTxs=0 Nov 01 17:29:18 105 bash[2257154]: INFO [2023-11-01|17:29:18.593] Committed state module=state height=16196200 txs=0 appHash=1030A7970FE55E09936A5EF80E5926D99943637656CECDA430D2D4C85A767E93 Nov 01 17:29:19 105 bash[2257154]: ERROR[2023-11-01|17:29:19.216] dialing failed (attempts: 3): auth failure: secret conn failed: read tcp 192.168.1.105:39642->174.127.69.33:26656: i/o timeout module=pex addr=18ed5af97328288e907a37c5561187c3aa06fe2b@174.127.69.33:26656 Nov 01 17:29:19 105 bash[2257154]: ERROR[2023-11-01|17:29:19.233] dialing failed (attempts: 1): auth failure: secret conn failed: read tcp 192.168.1.105:58580->174.127.69.23:26656: i/o timeout module=pex addr=4dec123d74b5c231d68250e2adafc9e7e5c027a8@174.127.69.23:26656 Nov 01 17:29:19 105 bash[2257154]: ERROR[2023-11-01|17:29:19.561] Stopping peer for error module=p2p peer="Peer{MConn{43.130.120.194:26656} 406ce7af245a4cfb7714403920e9cafca42aca46 out}" err=EOF Nov 01 17:29:24 105 bash[2257154]: INFO [2023-11-01|17:29:24.817] Executed block module=state height=16196201 validTxs=0 invalidTxs=0 Nov 01 17:29:24 105 bash[2257154]: INFO [2023-11-01|17:29:24.841] Committed state module=state height=16196201 txs=0 appHash=1030A7970FE55E09936A5EF80E5926D99943637656CECDA430D2D4C85A767E93 Nov 01 17:29:32 105 bash[2257154]: INFO [2023-11-01|17:29:32.167] Executed block module=state height=16196202 validTxs=2 invalidTxs=0 Nov 01 17:29:32 105 bash[2257154]: INFO [2023-11-01|17:29:32.193] Committed state module=state height=16196202 txs=2 appHash=6A783620745EC095CD013B7A990BE2EB84E73E968A5CBB2A0D66617D8E97D6EF Nov 01 17:29:38 105 bash[2257154]: INFO [2023-11-01|17:29:38.440] Executed block module=state height=16196203 validTxs=0 invalidTxs=0 Nov 01 17:29:38 105 bash[2257154]: INFO [2023-11-01|17:29:38.472] Committed state module=state height=16196203 txs=0 appHash=6A783620745EC095CD013B7A990BE2EB84E73E968A5CBB2A0D66617D8E97D6EF

It looks like my Heimdall is working fine.

cshintov commented 11 months ago

In the bor logs

Nov 01 17:07:09 105 bash[2271956]: [EROR] [11-01|17:07:09.804] Failed to fetch latest no-ack milestone  err="rpc error: code = DeadlineExceeded desc = context deadline exceeded"

means it's not able to talk to heimdallr. Check connection if heimdald and heimdallr are working fine.

presIey commented 11 months ago 
[one@105 ~]$ journalctl -fu heimdalld_rest
Nov 01 17:28:40 105 bash[2274764]: ERROR[2023-11-01|17:28:40.880] Cannot start REST server.                    module=rest-server Error="close tcp [::]:1317: use of closed network connection"
Nov 01 17:28:40 105 bash[2274764]: ERROR: close tcp [::]:1317: use of closed network connection
Nov 01 17:28:40 105 systemd[1]: heimdalld_rest.service: Deactivated successfully.
Nov 01 17:28:40 105 systemd[1]: Stopped Polygon Pos Light Client.
Nov 01 17:28:40 105 systemd[1]: Started Polygon Pos Light Client.
Nov 01 17:28:40 105 bash[2274794]: DEBUG[2023-11-01|17:28:40.943] Missing ETH RPC timeout or invalid value provided, falling back to default timeout=5s
Nov 01 17:28:40 105 bash[2274794]: DEBUG[2023-11-01|17:28:40.943] Missing BOR RPC timeout or invalid value provided, falling back to default timeout=5s
Nov 01 17:28:41 105 bash[2274794]: INFO [2023-11-01|17:28:41.036] Starting application REST service (chain-id: "137")... module=rest-server
Nov 01 17:28:41 105 bash[2274794]: INFO [2023-11-01|17:28:41.036] GRPC Server started                          module=gRPC-server addr=0.0.0.0:3132
Nov 01 17:28:41 105 bash[2274794]: INFO [2023-11-01|17:28:41.036] Starting RPC HTTP server on [::]:1317        module=rest-server
Nov 02 10:40:47 105 bash[2274794]: INFO [2023-11-02|10:40:47.248] Served RPC HTTP response                     module=rest-server method=GET url=/ status=404 duration=1 remoteAddr=[::1]:33118

[one@105 ~]$ curl -i localhost:1317
HTTP/1.1 404 Not Found
Content-Type: text/plain; charset=utf-8
X-Content-Type-Options: nosniff
X-Server-Time: 1698892847
Date: Thu, 02 Nov 2023 02:40:47 GMT
Content-Length: 19

404 page not found

Is there any problem with this connection status? Could you please provide other testing methods?

presIey commented 11 months ago 
[one@105 ~]$ curl localhost:26657/status
{
  "jsonrpc": "2.0",
  "id": "",
  "result": {
    "node_info": {
      "protocol_version": {
        "p2p": "7",
        "block": "10",
        "app": "0"
      },
      "id": "acdd4c22be1c1976f409b4c0141a2d8ab9916553",
      "listen_addr": "tcp://0.0.0.0:26656",
      "network": "heimdall-137",
      "version": "0.32.7",
      "channels": "4020212223303800",
      "moniker": "105",
      "other": {
        "tx_index": "on",
        "rpc_address": "tcp://127.0.0.1:26657"
      }
    },
    "sync_info": {
      "latest_block_hash": "520B384D58A148AD635B0732783DB9E32485AB1D9B4C99D423EB3F30BCA2E1BC",
      "latest_app_hash": "38A4DF1AC71185FA92B75BB969F3B91C1B2F0D4C7381B2E7B26E0650B9D09E5D",
      "latest_block_height": "16207076",
      "latest_block_time": "2023-11-02T05:32:18.810238625Z",
      "catching_up": false
    },
    "validator_info": {
      "address": "3795C8DD0D0CC853BE8F1F535ED18FB2037A8E7D",
      "pub_key": {
        "type": "tendermint/PubKeySecp256k1",
        "value": "BKZtif1clehxfS6yqpRWOdGei5s3PZxRZDYhMPUMOsZML9TpNui6h1AwHh9hDfkBwCJzj7ghEt+3tJYB6Ng11b4="
      },
      "voting_power": "0"
    }
  }
}[one@105 ~]$
presIey commented 11 months ago 
[one@105 ~]$curl localhost:1317/bor/span/11
{"height":"16207116","result":{"span_id":1,"start_block":256,"end_block":6655,"validator_set":{"validators":[{"ID":6,"startEpoch":0,"endEpoch":0,"nonce":1,"power":10000,"pubKey":"0x0447ed13442b485dd6990efc01d4297d798e90d3fa8467dd9c2f50ffe3238c8bf722f6c774f584b5fe91364b7b430c5a24fe57aca48665cf778030266f2c452bd9","signer":"0x0375b2fc7140977c9c76d45421564e354ed42277","last_updated":"","jailed":false,"accum":-60000},{"ID":7,"startEpoch":0,"endEpoch":0,"nonce":1,"power":10000,"pubKey":"0x043522a004012c9740703f676b95b5121edd7237fb0f182c3c45e7c7a77eaa67a20e6d0ac025d5bd96295bf95e2e875ab2a9da5c0e547b7d00ca7ede33c1b03893","signer":"0x42eefcda06ead475cde3731b8eb138e88cd0bac3","last_updated":"","jailed":false,"accum":10000},{"ID":1,"startEpoch":0,"endEpoch":0,"nonce":1,"power":10000,"pubKey":"0x043c53ea6e1964e0670dc9ac72b3224207885d4c5f08cabe1e1080c662fdec278e7e833798757cb5cf121447dcd02a15f010eb4aa87cceecb23daa4bf904112e77","signer":"0x5973918275c01f50555d44e92c9d9b353cadad54","last_updated":"","jailed":false,"accum":10000},{"ID":5,"startEpoch":0,"endEpoch":0,"nonce":1,"power":10000,"pubKey":"0x0479efe8c50b1f9923f48a467ecac0a64c2d6bcaa9ae67e135df84cac5aed5321f9cbb29c115f26dc84f2ef0e5fea29615848c79d690cb205cc10d688324ae8bce","signer":"0x7fcd58c2d53d980b247f1612fdba93e9a76193e6","last_updated":"","jailed":false,"accum":10000},{"ID":4,"startEpoch":0,"endEpoch":0,"nonce":1,"power":10000,"pubKey":"0x04b4e1d56b3429f7756452426be611e595debcb858d59f47d29ec9dd6e4b547dce1539f9b7144420bc309de496b70d6dc5f13345eee85e6b7fb332cd9f364ef12f","signer":"0xb702f1c9154ac9c08da247a8e30ee6f2f3373f41","last_updated":"","jailed":false,"accum":10000},{"ID":2,"startEpoch":0,"endEpoch":0,"nonce":1,"power":10000,"pubKey":"0x04d6b06c725f5410e4ccbd65906ece180364ebea4902b21232c1fb5892a76be7eec22480397d6bf653e9abe7ac50435ee472b59364fe78b17acb2be2116f92a76f","signer":"0xb8bb158b93c94ed35c1970d610d1e2b34e26652c","last_updated":"","jailed":false,"accum":10000},{"ID":3,"startEpoch":0,"endEpoch":0,"nonce":1,"power":10000,"pubKey":"0x040600efda73e1404b0c596e08c78c5ed51631fc173e5f39d21deeddd5712fcd7d6d440c53d211eb48b03063a05b2c0c0eb084053dfcf1c6540def705c8e028456","signer":"0xf84c74dea96df0ec22e11e7c33996c73fcc2d822","last_updated":"","jailed":false,"accum":10000}],"proposer":{"ID":6,"startEpoch":0,"endEpoch":0,"nonce":1,"power":10000,"pubKey":"0x0447ed13442b485dd6990efc01d4297d798e90d3fa8467dd9c2f50ffe3238c8bf722f6c774f584b5fe91364b7b430c5a24fe57aca48665cf778030266f2c452bd9","signer":"0x0375b2fc7140977c9c76d45421564e354ed42277","last_updated":"","jailed":false,"accum":-60000}},"selected_producers":[{"ID":6,"startEpoch":0,"endEpoch":0,"nonce":1,"power":10000,"pubKey":"0x0447ed13442b485dd6990efc01d4297d798e90d3fa8467dd9c2f50ffe3238c8bf722f6c774f584b5fe91364b7b430c5a24fe57aca48665cf778030266f2c452bd9","signer":"0x0375b2fc7140977c9c76d45421564e354ed42277","last_updated":"","jailed":false,"accum":-60000},{"ID":7,"startEpoch":0,"endEpoch":0,"nonce":1,"power":10000,"pubKey":"0x043522a004012c9740703f676b95b5121edd7237fb0f182c3c45e7c7a77eaa67a20e6d0ac025d5bd96295bf95e2e875ab2a9da5c0e547b7d00ca7ede33c1b03893","signer":"0x42eefcda06ead475cde3731b8eb138e88cd0bac3","last_updated":"","jailed":false,"accum":10000},{"ID":1,"startEpoch":0,"endEpoch":0,"nonce":1,"power":10000,"pubKey":"0x043c53ea6e1964e0670dc9ac72b3224207885d4c5f08cabe1e1080c662fdec278e7e833798757cb5cf121447dcd02a15f010eb4aa87cceecb23daa4bf904112e77","signer":"0x5973918275c01f50555d44e92c9d9b353cadad54","last_updated":"","jailed":false,"accum":10000},{"ID":5,"startEpoch":0,"endEpoch":0,"nonce":1,"power":10000,"pubKey":"0x0479efe8c50b1f9923f48a467ecac0a64c2d6bcaa9ae67e135df84cac5aed5321f9cbb29c115f26dc84f2ef0e5fea29615848c79d690cb205cc10d688324ae8bce","signer":"0x7fcd58c2d53d980b247f1612fdba93e9a76193e6","last_updated":"","jailed":false,"accum":10000},{"ID":4,"startEpoch":0,"endEpoch":0,"nonce":1,"power":10000,"pubKey":"0x04b4e1d56b3429f7756452426be611e595debcb858d59f47d29ec9dd6e4b547dce1539f9b7144420bc309de496b70d6dc5f13345eee85e6b7fb332cd9f364ef12f","signer":"0xb702f1c9154ac9c08da247a8e30ee6f2f3373f41","last_updated":"","jailed":false,"accum":10000},{"ID":2,"startEpoch":0,"endEpoch":0,"nonce":1,"power":10000,"pubKey":"0x04d6b06c725f5410e4ccbd65906ece180364ebea4902b21232c1fb5892a76be7eec22480397d6bf653e9abe7ac50435ee472b59364fe78b17acb2be2116f92a76f","signer":"0xb8bb158b93c94ed35c1970d610d1e2b34e26652c","last_updated":"","jailed":false,"accum":10000},{"ID":3,"startEpoch":0,"endEpoch":0,"nonce":1,"power":10000,"pubKey":"0x040600efda73e1404b0c596e08c78c5ed51631fc173e5f39d21deeddd5712fcd7d6d440c53d211eb48b03063a05b2c0c0eb084053dfcf1c6540def705c8e028456","signer":"0xf84c74dea96df0ec22e11e7c33996c73fcc2d822","last_updated":"","jailed":false,"accum":10000}],"bor_chain_id":"137"}}[one@105 ~]$
cshintov commented 11 months ago

There's port mismatch! 26657 is the correct one to pass to bor. That's where heimdallr is listening to.

presIey commented 11 months ago 
polygon bor

[one@105 node_scripts]$ cat start_erigon_polygon.sh
/home/one/erigon/build/bin/erigon   \
        --datadir=/mnt/data_lvmu2/polygon/erigon  \
        --chain=bor-mainnet \
        --snapshots=true   \
        --bor.heimdall=http://localhost:1317  \
        --bor.heimdallgRPC=http://localhost:3132 \
        --torrent.download.rate="500mb" \
        --txpool.api.addr=14006  \
        --port=30304 \
        --http  \
        --http.addr="0.0.0.0" \
        --http.port=14005 \
        --torrent.port=14008 \
        --http.api=eth,erigon,engine,web3,net,txpool \
        --private.api.addr="127.0.0.1:9195"  \
        --http.corsdomain '*' \
        --ws 

[one@105 node_scripts]$ cat start_heimdalld.sh
/home/one/go/bin/heimdalld   start   --home=/mnt/data_lvmu2/polygon/heimdall/home
[one@105 node_scripts]$ 

[one@105 node_scripts]$ cat start_heimdalld_rest.sh
/home/one/go/bin/heimdalld   rest-server   start  --chain-id=137   --home=/mnt/data_lvmu2/polygon/heimdall/home

[one@105 node_scripts]$ 

[one@105 node_scripts]$ cat /mnt/data_lvmu2/polygon/heimdall/home/config/
addrbook.json            config.toml              heimdall-config.toml     priv_validator_key.json  
app.toml                 genesis.json             node_key.json            
[one@105 node_scripts]$ cat /mnt/data_lvmu2/polygon/heimdall/home/config/app.toml 
# This is a TOML config file.
# For more information, see https://github.com/toml-lang/toml

##### main base config options #####

# The minimum gas prices a validator is willing to accept for processing a
# transaction. A transaction's fees must meet the minimum of any denomination
# specified in this config (e.g. 0.25token1;0.0001token2).
minimum-gas-prices = ""

# HaltHeight contains a non-zero block height at which a node will gracefully
# halt and shutdown that can be used to assist upgrades and testing.
#
# Note: State will not be committed on the corresponding height and any logs
# indicating such can be safely ignored.
halt-height = 0

# HaltTime contains a non-zero minimum block time (in Unix seconds) at which
# a node will gracefully halt and shutdown that can be used to assist upgrades
# and testing.
#
# Note: State will not be committed on the corresponding height and any logs
# indicating such can be safely ignored.
halt-time = 0
[one@105 node_scripts]$ 

[one@105 node_scripts]$ cat /mnt/data_lvmu2/polygon/heimdall/home/config/config.toml 
# This is a TOML config file.
# For more information, see https://github.com/toml-lang/toml

##### main base config options #####

# TCP or UNIX socket address of the ABCI application,
# or the name of an ABCI application compiled in with the Tendermint binary
proxy_app = "tcp://127.0.0.1:26658"

# A custom human readable name for this node
moniker = "105"

# If this node is many blocks behind the tip of the chain, FastSync
# allows them to catchup quickly by downloading blocks in parallel
# and verifying their commits
fast_sync = true

# Database backend: goleveldb | cleveldb | boltdb
# * goleveldb (github.com/syndtr/goleveldb - most popular implementation)
#   - pure go
#   - stable
# * cleveldb (uses levigo wrapper)
#   - fast
#   - requires gcc
#   - use cleveldb build tag (go build -tags cleveldb)
# * boltdb (uses etcd's fork of bolt - github.com/etcd-io/bbolt)
#   - EXPERIMENTAL
#   - may be faster is some use-cases (random reads - indexer)
#   - use boltdb build tag (go build -tags boltdb)
db_backend = "goleveldb"

# Database directory
db_dir = "data"

# Output level for logging, including package level options
log_level = "main:info,state:info,*:error"

# Output format: 'plain' (colored text) or 'json'
log_format = "plain"

##### additional base config options #####

# Path to the JSON file containing the initial validator set and other meta data
genesis_file = "config/genesis.json"

# Path to the JSON file containing the private key to use as a validator in the consensus protocol
priv_validator_key_file = "config/priv_validator_key.json"

# Path to the JSON file containing the last sign state of a validator
priv_validator_state_file = "data/priv_validator_state.json"

# TCP or UNIX socket address for Tendermint to listen on for
# connections from an external PrivValidator process
priv_validator_laddr = ""

# Path to the JSON file containing the private key to use for node authentication in the p2p protocol
node_key_file = "config/node_key.json"

# Mechanism to connect to the ABCI application: socket | grpc
abci = "socket"

# TCP or UNIX socket address for the profiling server to listen on
prof_laddr = "localhost:6060"

# If true, query the ABCI app on connecting to a new peer
# so the app can decide if we should keep the connection or not
filter_peers = false

##### advanced configuration options #####

##### rpc server configuration options #####
[rpc]

# TCP or UNIX socket address for the RPC server to listen on
laddr = "tcp://127.0.0.1:26657"

# A list of origins a cross-domain request can be executed from
# Default value '[]' disables cors support
# Use '["*"]' to allow any origin
cors_allowed_origins = []

# A list of methods the client is allowed to use with cross-domain requests
cors_allowed_methods = ["HEAD", "GET", "POST", ]

# A list of non simple headers the client is allowed to use with cross-domain requests
cors_allowed_headers = ["Origin", "Accept", "Content-Type", "X-Requested-With", "X-Server-Time", ]

# TCP or UNIX socket address for the gRPC server to listen on
# NOTE: This server only supports /broadcast_tx_commit
grpc_laddr = ""

# Maximum number of simultaneous connections.
# Does not include RPC (HTTP&WebSocket) connections. See max_open_connections
# If you want to accept a larger number than the default, make sure
# you increase your OS limits.
# 0 - unlimited.
# Should be < {ulimit -Sn} - {MaxNumInboundPeers} - {MaxNumOutboundPeers} - {N of wal, db and other open files}
# 1024 - 40 - 10 - 50 = 924 = ~900
grpc_max_open_connections = 900

# Activate unsafe RPC commands like /dial_seeds and /unsafe_flush_mempool
unsafe = false

# Maximum number of simultaneous connections (including WebSocket).
# Does not include gRPC connections. See grpc_max_open_connections
# If you want to accept a larger number than the default, make sure
# you increase your OS limits.
# 0 - unlimited.
# Should be < {ulimit -Sn} - {MaxNumInboundPeers} - {MaxNumOutboundPeers} - {N of wal, db and other open files}
# 1024 - 40 - 10 - 50 = 924 = ~900
max_open_connections = 900

# Maximum number of unique clientIDs that can /subscribe
# If you're using /broadcast_tx_commit, set to the estimated maximum number
# of broadcast_tx_commit calls per block.
max_subscription_clients = 100

# Maximum number of unique queries a given client can /subscribe to
# If you're using GRPC (or Local RPC client) and /broadcast_tx_commit, set to
# the estimated # maximum number of broadcast_tx_commit calls per block.
max_subscriptions_per_client = 5

# How long to wait for a tx to be committed during /broadcast_tx_commit.
# WARNING: Using a value larger than 10s will result in increasing the
# global HTTP write timeout, which applies to all connections and endpoints.
# See https://github.com/tendermint/tendermint/issues/3435
timeout_broadcast_tx_commit = "10s"

# Maximum size of request body, in bytes
max_body_bytes = 1000000

# Maximum size of request header, in bytes
max_header_bytes = 1048576

# The path to a file containing certificate that is used to create the HTTPS server.
# Migth be either absolute path or path related to tendermint's config directory.
# If the certificate is signed by a certificate authority,
# the certFile should be the concatenation of the server's certificate, any intermediates,
# and the CA's certificate.
# NOTE: both tls_cert_file and tls_key_file must be present for Tendermint to create HTTPS server. Otherwise, HTTP server is run.
tls_cert_file = ""

# The path to a file containing matching private key that is used to create the HTTPS server.
# Migth be either absolute path or path related to tendermint's config directory.
# NOTE: both tls_cert_file and tls_key_file must be present for Tendermint to create HTTPS server. Otherwise, HTTP server is run.
tls_key_file = ""

##### peer to peer configuration options #####
[p2p]

# Address to listen for incoming connections
laddr = "tcp://0.0.0.0:26656"

# Address to advertise to peers for them to dial
# If empty, will use the same port as the laddr,
# and will introspect on the listener or use UPnP
# to figure out the address.
external_address = ""

# Comma separated list of seed nodes to connect to
seeds = ""

# Comma separated list of nodes to keep persistent connections to
persistent_peers = ""

# UPNP port forwarding
upnp = true

# Path to address book
addr_book_file = "config/addrbook.json"

# Set true for strict address routability rules
# Set false for private or local networks
addr_book_strict = true

# Maximum number of inbound peers
max_num_inbound_peers = 1000

# Maximum number of outbound peers to connect to, excluding persistent peers
max_num_outbound_peers = 1000

# Time to wait before flushing messages out on the connection
flush_throttle_timeout = "500ms"

# Maximum size of a message packet payload, in bytes
max_packet_msg_payload_size = 1024

# Rate at which packets can be sent, in bytes/second
send_rate = 5120000

# Rate at which packets can be received, in bytes/second
recv_rate = 5120000

# Set true to enable the peer-exchange reactor
pex = true

# Seed mode, in which node constantly crawls the network and looks for
# peers. If another node asks it for addresses, it responds and disconnects.
#
# Does not work if the peer-exchange reactor is disabled.
seed_mode = false

# Comma separated list of peer IDs to keep private (will not be gossiped to other peers)
private_peer_ids = ""

# Toggle to disable guard against peers connecting from the same ip.
allow_duplicate_ip = false

# Peer connection configuration.
handshake_timeout = "20s"
dial_timeout = "3s"

##### mempool configuration options #####
[mempool]

recheck = true
broadcast = true
wal_dir = ""

# Maximum number of transactions in the mempool
size = 5000

# Limit the total size of all txs in the mempool.
# This only accounts for raw transactions (e.g. given 1MB transactions and
# max_txs_bytes=5MB, mempool will only accept 5 transactions).
max_txs_bytes = 1073741824

# Size of the cache (used to filter transactions we saw earlier) in transactions
cache_size = 100000

# Maximum size of a single transaction.
# NOTE: the max size of a tx transmitted over the network is {max_tx_bytes} + {amino overhead}.
max_tx_bytes = 1048576

##### fast sync configuration options #####
[fastsync]

# Fast Sync version to use:
#   1) "v0" (default) - the legacy fast sync implementation
#   2) "v1" - refactor of v0 version for better testability
version = "v0"

##### consensus configuration options #####
[consensus]

wal_file = "data/cs.wal/wal"

timeout_propose = "3s"
timeout_propose_delta = "500ms"
timeout_prevote = "1s"
timeout_prevote_delta = "500ms"
timeout_precommit = "1s"
timeout_precommit_delta = "500ms"
timeout_commit = "5s"

# Make progress as soon as we have all the precommits (as if TimeoutCommit = 0)
skip_timeout_commit = false

# EmptyBlocks mode and possible interval between empty blocks
create_empty_blocks = true
create_empty_blocks_interval = "0s"

# Reactor sleep duration parameters
peer_gossip_sleep_duration = "100ms"
peer_query_maj23_sleep_duration = "2s"

##### transactions indexer configuration options #####
[tx_index]

# What indexer to use for transactions
#
# Options:
#   1) "null"
#   2) "kv" (default) - the simplest possible indexer, backed by key-value storage (defaults to levelDB; see DBBackend).
indexer = "kv"

# Comma-separated list of tags to index (by default the only tag is "tx.hash")
#
# You can also index transactions by height by adding "tx.height" tag here.
#
# It's recommended to index only a subset of tags due to possible memory
# bloat. This is, of course, depends on the indexer's DB and the volume of
# transactions.
index_tags = ""

# When set to true, tells indexer to index all tags (predefined tags:
# "tx.hash", "tx.height" and all tags from DeliverTx responses).
#
# Note this may be not desirable (see the comment above). IndexTags has a
# precedence over IndexAllTags (i.e. when given both, IndexTags will be
# indexed).
index_all_tags = true

##### instrumentation configuration options #####
[instrumentation]

# When true, Prometheus metrics are served under /metrics on
# PrometheusListenAddr.
# Check out the documentation for the list of available metrics.
prometheus = false

# Address to listen for Prometheus collector(s) connections
prometheus_listen_addr = ":26660"

# Maximum number of simultaneous connections.
# If you want to accept a larger number than the default, make sure
# you increase your OS limits.
# 0 - unlimited.
max_open_connections = 5

# Instrumentation namespace
namespace = "tendermint"
[one@105 node_scripts]$ 

[one@105 node_scripts]$ cat /mnt/data_lvmu2/polygon/heimdall/home/config/heimdall-config.toml 
# This is a TOML config file.
# For more information, see https://github.com/toml-lang/toml

##### RPC and REST configs #####

# RPC endpoint for ethereum chain
eth_rpc_url = "http://192.168.1.104:10005"

# RPC endpoint for bor chain
bor_rpc_url = "http://localhost:14005"

# RPC endpoint for tendermint
tendermint_rpc_url = "http://0.0.0.0:26657"

# Polygon Sub Graph URL for self-heal mechanism (optional)
sub_graph_url = ""

#### Bridge configs ####

# Heimdall REST server endpoint, which is used by bridge
heimdall_rest_server = "http://0.0.0.0:1317"

# AMQP endpoint
amqp_url = "amqp://guest:guest@localhost:5672/"

## Poll intervals
checkpoint_poll_interval = "5m0s"
syncer_poll_interval = "1m0s"
noack_poll_interval = "16m50s"
clerk_poll_interval = "10s"
span_poll_interval = "1m0s"
milestone_poll_interval = "30s"
enable_self_heal = "false"
sh_state_synced_interval = "15m0s"
sh_stake_update_interval = "3h0m0s"
sh_max_depth_duration = "1h0m0s"

#### gas limits ####
main_chain_gas_limit = "5000000"

#### gas price ####
main_chain_max_gas_price = "400000000000"

##### Timeout Config #####
no_ack_wait_time = "30m0s"

##### chain - newSelectionAlgoHeight depends on this #####
chain = "mainnet"
[one@105 node_scripts]$

So how should I modify my configuration?

presIey commented 11 months ago

I made sure I configured it according to the official instructions.

presIey commented 11 months ago

Please help me, I have been troubled by this problem for a long time, thank you very much 🙏!

cshintov commented 11 months ago

Check your heimdalld config.toml

[rpc]

# TCP or UNIX socket address for the RPC server to listen on
laddr = "tcp://[127.0.0.1:26657](http://127.0.0.1:26657/)"

You only need this for the erigon.

--bor.heimdall=http://localhost:26657

Also pass this to heimdall rest server as

      --laddr tcp://0.0.0.0:26657
cshintov commented 11 months ago

Also I suggest read the docs thoroughly and if you are in doubt, head over to erigon/polygon discord and directly ask your doubts there. I believe that's a better place for these kinds of questions.

cshintov commented 11 months ago

Also, consult these notes: https://github.com/davidedg/polygon-node-notes, it might help.

thanks to @davidedg

IvanLazarevsky commented 11 months ago

Having the same problem. Switching bor.heimdall port doesn't work, there are actually no such endpoints as no-ack milestone at 26657 so @cshintov answer is wrong.

IvanLazarevsky commented 11 months ago

It seems there is a mess with timeouts, because endpoints like http://localhost:1317/milestone/latest are easy to curl: {"height":"16352560","result":{"proposer":"0x1efecb61a2f80aa34d3b9218b564a64d05946290","start_block":49890918,"end_block":49890952,"hash":"0xe476a1760d42bd69721ee29dbeb03c9a532eeb7cc3bbc796b46714d998b47128","bor_chain_id":"137","milestone_id":"b775fe0a-eba9-4b2a-b421-46f6e0eacaf0 - 0xbeb03c9a532eeb7cc3bbc796b46714d998b47128","timestamp":1699873270}}(