search

NethermindEth / nethermind

A robust execution client for Ethereum node operators.
https://nethermind.io/nethermind-client
GNU General Public License v3.0
1.23k stars 428 forks source link

Receipt not available for 'From' block '11052984' #5230

Closed smasher816 closed 1 year ago

smasher816 commented 1 year ago

Running the following build:

Version: 1.16.1+faf9742e
Commit: faf9742ef2d1a533bc5e70b69b1789215fe306c5
Build Date: 2023-01-31 15:10:05Z
OS: Linux x64
Runtime: .NET 7.0.2

With the following config:

ExecStart=/opt/nethermind/src/Nethermind/Nethermind.Runner/bin/Release/net7.0/Nethermind.Runner \
  --datadir /var/lib/nethermind \
  --JsonRpc.JwtSecretFile /var/lib/nethermind/jwtsecret \
  --Sync.SnapSync true \
  --Network.ActivePeersMaxCount 10 \
  --Metrics.Enabled true \
  --Metrics.ExposePort 6060 \
  --Metrics.IntervalSeconds 10000

And receiving the following error:

Feb 02 19:11:02 hoenir Nethermind.Runner[448]: 2023-02-02 19:11:02.4220|Executing JSON RPC call eth_getLogs with params [{
Feb 02 19:11:02 hoenir Nethermind.Runner[448]:   "topics": [
Feb 02 19:11:02 hoenir Nethermind.Runner[448]:     "0x649bbc62d0e31342afea4e5cd82d4049e7e1ee912fc0889aa790803be39038c5"
Feb 02 19:11:02 hoenir Nethermind.Runner[448]:   ],
Feb 02 19:11:02 hoenir Nethermind.Runner[448]:   "fromBlock": "0xa8a7b8",
Feb 02 19:11:02 hoenir Nethermind.Runner[448]:   "toBlock": "0xa8a7b9",
Feb 02 19:11:02 hoenir Nethermind.Runner[448]:   "address": [
Feb 02 19:11:02 hoenir Nethermind.Runner[448]:     "0x00000000219ab540356cBB839Cbe05303d7705Fa"
Feb 02 19:11:02 hoenir Nethermind.Runner[448]:   ]
Feb 02 19:11:02 hoenir Nethermind.Runner[448]: }]
Feb 02 19:11:02 hoenir Nethermind.Runner[448]: 2023-02-02 19:11:02.4220|Error when handling ID 976, eth_getLogs({
Feb 02 19:11:02 hoenir Nethermind.Runner[448]:   "topics": [
Feb 02 19:11:02 hoenir Nethermind.Runner[448]:     "0x649bbc62d0e31342afea4e5cd82d4049e7e1ee912fc0889aa790803be39038c5"
Feb 02 19:11:02 hoenir Nethermind.Runner[448]:   ],
Feb 02 19:11:02 hoenir Nethermind.Runner[448]:   "fromBlock": "0xa8a7b8",
Feb 02 19:11:02 hoenir Nethermind.Runner[448]:   "toBlock": "0xa8a7b9",
Feb 02 19:11:02 hoenir Nethermind.Runner[448]:   "address": [
Feb 02 19:11:02 hoenir Nethermind.Runner[448]:     "0x00000000219ab540356cBB839Cbe05303d7705Fa"
Feb 02 19:11:02 hoenir Nethermind.Runner[448]:   ]
Feb 02 19:11:02 hoenir Nethermind.Runner[448]: }) | {"jsonrpc":"2.0","error":{"code":-32001,"message":"Receipt not available for 'From' block '11052984'."},"id":976}
Feb 02 19:11:03 hoenir Nethermind.Runner[448]: 2023-02-02 19:11:03.9032|Executing JSON RPC call eth_getBlockByNumber with params [0xfc6d95, False]
Feb 02 19:11:04 hoenir Nethermind.Runner[448]: 2023-02-02 19:11:04.4251|Executing JSON RPC call eth_getLogs with params [{
Feb 02 19:11:04 hoenir Nethermind.Runner[448]:   "topics": [
Feb 02 19:11:04 hoenir Nethermind.Runner[448]:     "0x649bbc62d0e31342afea4e5cd82d4049e7e1ee912fc0889aa790803be39038c5"
Feb 02 19:11:04 hoenir Nethermind.Runner[448]:   ],
Feb 02 19:11:04 hoenir Nethermind.Runner[448]:   "fromBlock": "0xa8a7b8",
Feb 02 19:11:04 hoenir Nethermind.Runner[448]:   "toBlock": "0xa8a7b9",
Feb 02 19:11:04 hoenir Nethermind.Runner[448]:   "address": [
Feb 02 19:11:04 hoenir Nethermind.Runner[448]:     "0x00000000219ab540356cBB839Cbe05303d7705Fa"
Feb 02 19:11:04 hoenir Nethermind.Runner[448]:   ]
Feb 02 19:11:04 hoenir Nethermind.Runner[448]: }]

When originally syncing there were no errors, however after doing a "systemctl restart nethermind" I begin to see the above.

asdacap commented 1 year ago

If you run it with --Bloom.Index false does it work? This check if its an issue with bloom or receipt db..

kamilchodola commented 1 year ago

@smasher816 Hi! :)

Is Your node synced? Can You send more logs?

kamilchodola commented 1 year ago

@smasher816 I'm checking also Your version:

Version: 1.16.1+faf9742e Commit: faf9742ef2d1a533bc5e70b69b1789215fe306c5

Seems like You have a version built on top of master branch from our repository, right? This commit seems to not be included in official version.

smasher816 commented 1 year ago

@asdacap Adding --Bloom.Index false did not change the behavior. @kamilchodola The node was fully synced on jan31 before the restart

@smartprogrammer93 I have moved to the following build and still see the same issue. Do I need to somehow fix my existing db?

Version: 1.16.1+7a462db6
Commit: 7a462db658d9c50bdeeee372a81291a4a5146dc8
Build Date: 2023-02-03 14:32:02Z
OS: Linux x64
Runtime: .NET 7.0.2
Feb 03 06:42:11 hoenir Nethermind.Runner[5662]: 2023-02-03 06:42:11.4313|Processed   16548921 |      355ms of  11,507ms, mgasps   67.75 total   73.24, tps  847.98 total  818.79, bps    2.82 total    2.40, recv queue 0, proc queue 0
Feb 03 06:42:11 hoenir Nethermind.Runner[5662]: 2023-02-03 06:42:11.4313|Valid. Result of a new payload: 16548921 (0xd4335952c112b03ab28db1923911115fc990ebbd3018c9e956e4adf6cbccca81).
Feb 03 06:42:11 hoenir Nethermind.Runner[5662]: 2023-02-03 06:42:11.4407|Received: ForkchoiceState: (HeadBlockHash: 0xd4335952c112b03ab28db1923911115fc990ebbd3018c9e956e4adf6cbccca81, SafeBlockHash: 0xea168d2e3c53e9a20bd4456421dcedc7252dc6492b7be1e769d7e51ac946ed1e, FinalizedBlockHash: 0xe4a6e31309d9c2198b90b74025ce7cf0a309820b6e268872fdf53cc140721ee0)
Feb 03 06:42:11 hoenir Nethermind.Runner[5662]: 2023-02-03 06:42:11.4407|FCU - block 16548921 (0xd43359...ccca81) was processed.
Feb 03 06:42:11 hoenir Nethermind.Runner[5662]: 2023-02-03 06:42:11.4407|Block 0xd4335952c112b03ab28db1923911115fc990ebbd3018c9e956e4adf6cbccca81 was set as head.
Feb 03 06:42:11 hoenir Nethermind.Runner[5662]: 2023-02-03 06:42:11.4407|Valid. Request: ForkchoiceState: (HeadBlockHash: 0xd4335952c112b03ab28db1923911115fc990ebbd3018c9e956e4adf6cbccca81, SafeBlockHash: 0xea168d2e3c53e9a20bd4456421dcedc7252dc6492b7be1e769d7e51ac946ed1e, FinalizedBlockHash: 0xe4a6e31309d9c2198b90b74025ce7cf0a309820b6e268872fdf53cc140721ee0) .
Feb 03 06:42:13 hoenir Nethermind.Runner[5662]: 2023-02-03 06:42:13.2109|Executing JSON RPC call eth_getLogs with params [{
Feb 03 06:42:13 hoenir Nethermind.Runner[5662]:   "topics": [
Feb 03 06:42:13 hoenir Nethermind.Runner[5662]:     "0x649bbc62d0e31342afea4e5cd82d4049e7e1ee912fc0889aa790803be39038c5"
Feb 03 06:42:13 hoenir Nethermind.Runner[5662]:   ],
Feb 03 06:42:13 hoenir Nethermind.Runner[5662]:   "fromBlock": "0xa8a7b8",
Feb 03 06:42:13 hoenir Nethermind.Runner[5662]:   "toBlock": "0xa8a7b9",
Feb 03 06:42:13 hoenir Nethermind.Runner[5662]:   "address": [
Feb 03 06:42:13 hoenir Nethermind.Runner[5662]:     "0x00000000219ab540356cBB839Cbe05303d7705Fa"
Feb 03 06:42:13 hoenir Nethermind.Runner[5662]:   ]
Feb 03 06:42:13 hoenir Nethermind.Runner[5662]: }]
Feb 03 06:42:13 hoenir Nethermind.Runner[5662]: 2023-02-03 06:42:13.2109|Error when handling ID 35820, eth_getLogs({
Feb 03 06:42:13 hoenir Nethermind.Runner[5662]:   "topics": [
Feb 03 06:42:13 hoenir Nethermind.Runner[5662]:     "0x649bbc62d0e31342afea4e5cd82d4049e7e1ee912fc0889aa790803be39038c5"
Feb 03 06:42:13 hoenir Nethermind.Runner[5662]:   ],
Feb 03 06:42:13 hoenir Nethermind.Runner[5662]:   "fromBlock": "0xa8a7b8",
Feb 03 06:42:13 hoenir Nethermind.Runner[5662]:   "toBlock": "0xa8a7b9",
Feb 03 06:42:13 hoenir Nethermind.Runner[5662]:   "address": [
Feb 03 06:42:13 hoenir Nethermind.Runner[5662]:     "0x00000000219ab540356cBB839Cbe05303d7705Fa"
Feb 03 06:42:13 hoenir Nethermind.Runner[5662]:   ]
Feb 03 06:42:13 hoenir Nethermind.Runner[5662]: }) | {"jsonrpc":"2.0","error":{"code":-32001,"message":"Receipt not available for 'From' block '11052984'."},"id":35820}
Feb 03 06:42:15 hoenir Nethermind.Runner[5662]: 2023-02-03 06:42:15.0764|Executing JSON RPC call eth_getBlockByNumber with params [0xfc7af3, False]
Feb 03 06:42:15 hoenir Nethermind.Runner[5662]: 2023-02-03 06:42:15.2214|Executing JSON RPC call eth_getLogs with params [{
Feb 03 06:42:15 hoenir Nethermind.Runner[5662]:   "topics": [
Feb 03 06:42:15 hoenir Nethermind.Runner[5662]:     "0x649bbc62d0e31342afea4e5cd82d4049e7e1ee912fc0889aa790803be39038c5"
Feb 03 06:42:15 hoenir Nethermind.Runner[5662]:   ],
Feb 03 06:42:15 hoenir Nethermind.Runner[5662]:   "fromBlock": "0xa8a7b8",
Feb 03 06:42:15 hoenir Nethermind.Runner[5662]:   "toBlock": "0xa8a7b9",
Feb 03 06:42:15 hoenir Nethermind.Runner[5662]:   "address": [
Feb 03 06:42:15 hoenir Nethermind.Runner[5662]:     "0x00000000219ab540356cBB839Cbe05303d7705Fa"
Feb 03 06:42:15 hoenir Nethermind.Runner[5662]:   ]
Feb 03 06:42:15 hoenir Nethermind.Runner[5662]: }]
Feb 03 06:42:15 hoenir Nethermind.Runner[5662]: 2023-02-03 06:42:15.2220|Error when handling ID 35822, eth_getLogs({
Feb 03 06:42:15 hoenir Nethermind.Runner[5662]:   "topics": [
Feb 03 06:42:15 hoenir Nethermind.Runner[5662]:     "0x649bbc62d0e31342afea4e5cd82d4049e7e1ee912fc0889aa790803be39038c5"
Feb 03 06:42:15 hoenir Nethermind.Runner[5662]:   ],
Feb 03 06:42:15 hoenir Nethermind.Runner[5662]:   "fromBlock": "0xa8a7b8",
Feb 03 06:42:15 hoenir Nethermind.Runner[5662]:   "toBlock": "0xa8a7b9",
Feb 03 06:42:15 hoenir Nethermind.Runner[5662]:   "address": [
Feb 03 06:42:15 hoenir Nethermind.Runner[5662]:     "0x00000000219ab540356cBB839Cbe05303d7705Fa"
Feb 03 06:42:15 hoenir Nethermind.Runner[5662]:   ]
Feb 03 06:42:15 hoenir Nethermind.Runner[5662]: }) | {"jsonrpc":"2.0","error":{"code":-32001,"message":"Receipt not available for 'From' block '11052984'."},"id":35822}
Feb 03 06:42:17 hoenir Nethermind.Runner[5662]: 2023-02-03 06:42:17.2244|Executing JSON RPC call eth_getLogs with params [{
Feb 03 06:42:17 hoenir Nethermind.Runner[5662]:   "topics": [
Feb 03 06:42:17 hoenir Nethermind.Runner[5662]:     "0x649bbc62d0e31342afea4e5cd82d4049e7e1ee912fc0889aa790803be39038c5"
Feb 03 06:42:17 hoenir Nethermind.Runner[5662]:   ],
Feb 03 06:42:17 hoenir Nethermind.Runner[5662]:   "fromBlock": "0xa8a7b8",
Feb 03 06:42:17 hoenir Nethermind.Runner[5662]:   "toBlock": "0xa8a7b9",
Feb 03 06:42:17 hoenir Nethermind.Runner[5662]:   "address": [
Feb 03 06:42:17 hoenir Nethermind.Runner[5662]:     "0x00000000219ab540356cBB839Cbe05303d7705Fa"
Feb 03 06:42:17 hoenir Nethermind.Runner[5662]:   ]
Feb 03 06:42:17 hoenir Nethermind.Runner[5662]: }]
Feb 03 06:42:17 hoenir Nethermind.Runner[5662]: 2023-02-03 06:42:17.2244|Error when handling ID 35823, eth_getLogs({
Feb 03 06:42:17 hoenir Nethermind.Runner[5662]:   "topics": [
Feb 03 06:42:17 hoenir Nethermind.Runner[5662]:     "0x649bbc62d0e31342afea4e5cd82d4049e7e1ee912fc0889aa790803be39038c5"
Feb 03 06:42:17 hoenir Nethermind.Runner[5662]:   ],
Feb 03 06:42:17 hoenir Nethermind.Runner[5662]:   "fromBlock": "0xa8a7b8",
Feb 03 06:42:17 hoenir Nethermind.Runner[5662]:   "toBlock": "0xa8a7b9",
Feb 03 06:42:17 hoenir Nethermind.Runner[5662]:   "address": [
Feb 03 06:42:17 hoenir Nethermind.Runner[5662]:     "0x00000000219ab540356cBB839Cbe05303d7705Fa"
Feb 03 06:42:17 hoenir Nethermind.Runner[5662]:   ]
Feb 03 06:42:17 hoenir Nethermind.Runner[5662]: }) | {"jsonrpc":"2.0","error":{"code":-32001,"message":"Receipt not available for 'From' block '11052984'."},"id":35823}
Feb 03 06:42:19 hoenir Nethermind.Runner[5662]: 2023-02-03 06:42:19.2270|Executing JSON RPC call eth_getLogs with params [{
Feb 03 06:42:19 hoenir Nethermind.Runner[5662]:   "topics": [
Feb 03 06:42:19 hoenir Nethermind.Runner[5662]:     "0x649bbc62d0e31342afea4e5cd82d4049e7e1ee912fc0889aa790803be39038c5"
Feb 03 06:42:19 hoenir Nethermind.Runner[5662]:   ],
Feb 03 06:42:19 hoenir Nethermind.Runner[5662]:   "fromBlock": "0xa8a7b8",
Feb 03 06:42:19 hoenir Nethermind.Runner[5662]:   "toBlock": "0xa8a7b9",
Feb 03 06:42:19 hoenir Nethermind.Runner[5662]:   "address": [
Feb 03 06:42:19 hoenir Nethermind.Runner[5662]:     "0x00000000219ab540356cBB839Cbe05303d7705Fa"
Feb 03 06:42:19 hoenir Nethermind.Runner[5662]:   ]
Feb 03 06:42:19 hoenir Nethermind.Runner[5662]: }]
Feb 03 06:42:19 hoenir Nethermind.Runner[5662]: 2023-02-03 06:42:19.2270|Error when handling ID 35824, eth_getLogs({
Feb 03 06:42:19 hoenir Nethermind.Runner[5662]:   "topics": [
Feb 03 06:42:19 hoenir Nethermind.Runner[5662]:     "0x649bbc62d0e31342afea4e5cd82d4049e7e1ee912fc0889aa790803be39038c5"
Feb 03 06:42:19 hoenir Nethermind.Runner[5662]:   ],
Feb 03 06:42:19 hoenir Nethermind.Runner[5662]:   "fromBlock": "0xa8a7b8",
Feb 03 06:42:19 hoenir Nethermind.Runner[5662]:   "toBlock": "0xa8a7b9",
Feb 03 06:42:19 hoenir Nethermind.Runner[5662]:   "address": [
Feb 03 06:42:19 hoenir Nethermind.Runner[5662]:     "0x00000000219ab540356cBB839Cbe05303d7705Fa"
Feb 03 06:42:19 hoenir Nethermind.Runner[5662]:   ]
Feb 03 06:42:19 hoenir Nethermind.Runner[5662]: }) | {"jsonrpc":"2.0","error":{"code":-32001,"message":"Receipt not available for 'From' block '11052984'."},"id":35824}
qFeb 03 06:42:21 hoenir Nethermind.Runner[5662]: 2023-02-03 06:42:21.2299|Executing JSON RPC call eth_getLogs with params [{
Feb 03 06:42:21 hoenir Nethermind.Runner[5662]:   "topics": [
Feb 03 06:42:21 hoenir Nethermind.Runner[5662]:     "0x649bbc62d0e31342afea4e5cd82d4049e7e1ee912fc0889aa790803be39038c5"
Feb 03 06:42:21 hoenir Nethermind.Runner[5662]:   ],
Feb 03 06:42:21 hoenir Nethermind.Runner[5662]:   "fromBlock": "0xa8a7b8",
Feb 03 06:42:21 hoenir Nethermind.Runner[5662]:   "toBlock": "0xa8a7b9",
Feb 03 06:42:21 hoenir Nethermind.Runner[5662]:   "address": [
Feb 03 06:42:21 hoenir Nethermind.Runner[5662]:     "0x00000000219ab540356cBB839Cbe05303d7705Fa"
Feb 03 06:42:21 hoenir Nethermind.Runner[5662]:   ]
Feb 03 06:42:21 hoenir Nethermind.Runner[5662]: }]
Feb 03 06:42:21 hoenir Nethermind.Runner[5662]: 2023-02-03 06:42:21.2299|Error when handling ID 35825, eth_getLogs({
Feb 03 06:42:21 hoenir Nethermind.Runner[5662]:   "topics": [
Feb 03 06:42:21 hoenir Nethermind.Runner[5662]:     "0x649bbc62d0e31342afea4e5cd82d4049e7e1ee912fc0889aa790803be39038c5"
Feb 03 06:42:21 hoenir Nethermind.Runner[5662]:   ],
Feb 03 06:42:21 hoenir Nethermind.Runner[5662]:   "fromBlock": "0xa8a7b8",
Feb 03 06:42:21 hoenir Nethermind.Runner[5662]:   "toBlock": "0xa8a7b9",
Feb 03 06:42:21 hoenir Nethermind.Runner[5662]:   "address": [
Feb 03 06:42:21 hoenir Nethermind.Runner[5662]:     "0x00000000219ab540356cBB839Cbe05303d7705Fa"
Feb 03 06:42:21 hoenir Nethermind.Runner[5662]:   ]
Feb 03 06:42:21 hoenir Nethermind.Runner[5662]: }) | {"jsonrpc":"2.0","error":{"code":-32001,"message":"Receipt not available for 'From' block '11052984'."},"id":35825}
Feb 03 06:42:23 hoenir Nethermind.Runner[5662]: 2023-02-03 06:42:23.2325|Executing JSON RPC call eth_getLogs with params [{
Feb 03 06:42:23 hoenir Nethermind.Runner[5662]:   "topics": [
Feb 03 06:42:23 hoenir Nethermind.Runner[5662]:     "0x649bbc62d0e31342afea4e5cd82d4049e7e1ee912fc0889aa790803be39038c5"
Feb 03 06:42:23 hoenir Nethermind.Runner[5662]:   ],
Feb 03 06:42:23 hoenir Nethermind.Runner[5662]:   "fromBlock": "0xa8a7b8",
Feb 03 06:42:23 hoenir Nethermind.Runner[5662]:   "toBlock": "0xa8a7b9",
Feb 03 06:42:23 hoenir Nethermind.Runner[5662]:   "address": [
Feb 03 06:42:23 hoenir Nethermind.Runner[5662]:     "0x00000000219ab540356cBB839Cbe05303d7705Fa"
Feb 03 06:42:23 hoenir Nethermind.Runner[5662]:   ]
Feb 03 06:42:23 hoenir Nethermind.Runner[5662]: }]
Feb 03 06:42:23 hoenir Nethermind.Runner[5662]: 2023-02-03 06:42:23.2325|Error when handling ID 35826, eth_getLogs({
Feb 03 06:42:23 hoenir Nethermind.Runner[5662]:   "topics": [
Feb 03 06:42:23 hoenir Nethermind.Runner[5662]:     "0x649bbc62d0e31342afea4e5cd82d4049e7e1ee912fc0889aa790803be39038c5"
Feb 03 06:42:23 hoenir Nethermind.Runner[5662]:   ],
Feb 03 06:42:23 hoenir Nethermind.Runner[5662]:   "fromBlock": "0xa8a7b8",
Feb 03 06:42:23 hoenir Nethermind.Runner[5662]:   "toBlock": "0xa8a7b9",
Feb 03 06:42:23 hoenir Nethermind.Runner[5662]:   "address": [
Feb 03 06:42:23 hoenir Nethermind.Runner[5662]:     "0x00000000219ab540356cBB839Cbe05303d7705Fa"
Feb 03 06:42:23 hoenir Nethermind.Runner[5662]:   ]
Feb 03 06:42:23 hoenir Nethermind.Runner[5662]: }) | {"jsonrpc":"2.0","error":{"code":-32001,"message":"Receipt not available for 'From' block '11052984'."},"id":35826}
smartprogrammer93 commented 1 year ago

@smasher816 you need to resync from master or from a stable release

smasher816 commented 1 year ago

Is the best way to do that simply to remove the nethermind_db dir? Or is there a flag to force resync?

smartprogrammer93 commented 1 year ago

removing the directory is the correct way @smasher816

smasher816 commented 1 year ago

@smartprogrammer93 I have reverted to a stable build and removed the dir, however I am now getting a slightly different error "11052985 could not be found". It seems like some reference is still held even after the rm. I can see the resync of beacon headers is being performed.

Version: 1.16.1+644fe89f
Commit: 644fe89f94d589a2e7492c39ac1d40888974a571
Build Date: 2023-02-03 14:51:27Z
OS: Linux x64
Runtime: .NET 7.0.2
$ sudo systemctl stop nethermind
$ sudo rm -rf /var/lib/nethermind/nethermind_db/
$ sudo systemctl start nethermind
$ journalctl -fu nethermind
Feb 03 06:53:00 hoenir Nethermind.Runner[6938]: 2023-02-03 06:53:00.5402|Beacon Numbers resolved, level = 0, header = 0, body = 0
Feb 03 06:53:00 hoenir Nethermind.Runner[6938]: 2023-02-03 06:53:00.6941|Started monitoring for the group: nethermind, instance: Mainnet
Feb 03 06:53:00 hoenir Nethermind.Runner[6938]: 2023-02-03 06:53:00.7040|Starting Merge block producer & sealer
Feb 03 06:53:01 hoenir Nethermind.Runner[6938]: 2023-02-03 06:53:01.1286|Peers | with known best block: 0 | all: 0 |
Feb 03 06:53:01 hoenir Nethermind.Runner[6938]: 2023-02-03 06:53:01.1286|Waiting for peers... 1s
Feb 03 06:53:01 hoenir Nethermind.Runner[6938]: 2023-02-03 06:53:01.1353|Sync peers - Initialized: 0 | All: 0 | Max: 10
Feb 03 06:53:01 hoenir Nethermind.Runner[6938]: 2023-02-03 06:53:01.5847|Connected to 4 bootnodes, 4 trusted/persisted nodes
Feb 03 06:53:02 hoenir Nethermind.Runner[6938]: 2023-02-03 06:53:02.1369|Waiting for peers... 2s
Feb 03 06:53:03 hoenir Nethermind.Runner[6938]: 2023-02-03 06:53:03.1362|Waiting for peers... 3s
Feb 03 06:53:04 hoenir Nethermind.Runner[6938]: 2023-02-03 06:53:04.1363|Waiting for peers... 4s
Feb 03 06:53:05 hoenir Nethermind.Runner[6938]: 2023-02-03 06:53:05.1366|Waiting for peers... 5s
...
Feb 03 06:53:06 hoenir Nethermind.Runner[6938]: ======================== Nethermind initialization completed ========================
Feb 03 06:53:06 hoenir Nethermind.Runner[6938]: This node    : enode://3d4655b0bda98954b3897ca6a52e17b867c443533e912ec08118d98c278cc688fde449c284b261bae48f5720a57055e3d7ba77664da8e9ec63bb8f700681773f@201.141.100.36:30303
Feb 03 06:53:06 hoenir Nethermind.Runner[6938]: RPC modules  : Eth, Health, Net, Parity, Personal, Proof, Rpc, Subscribe, Trace, TxPool, Web3
Feb 03 06:53:06 hoenir Nethermind.Runner[6938]: Node address : 0x37fbccd6322681b45b1d9ec5fbf50da0e4ef84aa (do not use as an account)
Feb 03 06:53:06 hoenir Nethermind.Runner[6938]: Mem est tx   :     82MB
Feb 03 06:53:06 hoenir Nethermind.Runner[6938]: Mem est DB   :     17MB
Feb 03 06:53:06 hoenir Nethermind.Runner[6938]: JSON RPC     : http://127.0.0.1:8545 ; http://localhost:8551
Feb 03 06:53:06 hoenir Nethermind.Runner[6938]: Genesis hash : 0xd4e56740f876aef8c010b86a40d5f56745a118d0906a34e69aec8c0db1cb8fa3
Feb 03 06:53:06 hoenir Nethermind.Runner[6938]: External IP  : 201.141.100.36
Feb 03 06:53:06 hoenir Nethermind.Runner[6938]: Ethereum     : tcp://201.141.100.36:30303
Feb 03 06:53:06 hoenir Nethermind.Runner[6938]: Discovery    : udp://201.141.100.36:30303
Feb 03 06:53:06 hoenir Nethermind.Runner[6938]: Client id    : Nethermind/v1.16.1+644fe89f/linux-x64/dotnet7.0.2
Feb 03 06:53:06 hoenir Nethermind.Runner[6938]: Chainspec    : chainspec/foundation.json
Feb 03 06:53:06 hoenir Nethermind.Runner[6938]: Chain ID     : Mainnet
Feb 03 06:53:06 hoenir Nethermind.Runner[6938]: Chain head   : 0
Feb 03 06:53:06 hoenir Nethermind.Runner[6938]: =====================================================================================
...
Feb 03 06:56:56 hoenir Nethermind.Runner[6938]: 2023-02-03 06:56:56.4500|Error when handling ID 36622, eth_getLogs({
Feb 03 06:56:56 hoenir Nethermind.Runner[6938]:   "topics": [
Feb 03 06:56:56 hoenir Nethermind.Runner[6938]:     "0x649bbc62d0e31342afea4e5cd82d4049e7e1ee912fc0889aa790803be39038c5"
Feb 03 06:56:56 hoenir Nethermind.Runner[6938]:   ],
Feb 03 06:56:56 hoenir Nethermind.Runner[6938]:   "fromBlock": "0xa8a7b8",
Feb 03 06:56:56 hoenir Nethermind.Runner[6938]:   "toBlock": "0xa8a7b9",
Feb 03 06:56:56 hoenir Nethermind.Runner[6938]:   "address": [
Feb 03 06:56:56 hoenir Nethermind.Runner[6938]:     "0x00000000219ab540356cBB839Cbe05303d7705Fa"
Feb 03 06:56:56 hoenir Nethermind.Runner[6938]:   ]
Feb 03 06:56:56 hoenir Nethermind.Runner[6938]: }) | {"jsonrpc":"2.0","error":{"code":-32001,"message":"11052985 could not be found"},"id":36622}
Feb 03 06:56:57 hoenir Nethermind.Runner[6938]: 2023-02-03 06:56:57.2013|Old Headers  16469000 / 16469000 | queue     0 | current     0.00bps | total 38425104993.00bps
Feb 03 06:56:57 hoenir Nethermind.Runner[6938]: 2023-02-03 06:56:57.2013|Beacon Headers from block 16469001 to block 16548960 | 79959 / 79960 | queue     0 | current     0.00bps | total 291868765432.10bps
Feb 03 06:56:58 hoenir Nethermind.Runner[6938]: 2023-02-03 06:56:58.1994|Old Headers  16469000 / 16469000 | queue     0 | current     0.00bps | total 38425104993.00bps
Feb 03 06:56:58 hoenir Nethermind.Runner[6938]: 2023-02-03 06:56:58.1994|Beacon Headers from block 16469001 to block 16548960 | 79959 / 79960 | queue     0 | current     0.00bps | total 291868765432.10bps
smartprogrammer93 commented 1 year ago

it is normal to get the error while the node is still syncing @smasher816

kamilchodola commented 1 year ago

@smasher816 CL already started asking about this block and because regression caused that this receipt was not persisted well and You are resyncing, then You will be getting this error until You will sync receipts now. This is because I guess You were not resyncing CL in the same time. But this is not a problem - I have synced a node with exact same setup and it went well.

smasher816 commented 1 year ago

@kamilchodola you are right. teku was requesting the logs from the bad block (11052985). After restarting teku the eth_getLogs errors stopped being repeated in the nethermind output. I'll see what happens once the full resync finishes.

Feb 04 06:47:59 hoenir teku[452]: 06:47:59.100 INFO  - Syncing     *** Target slot: 5724838, Head slot: 5724837, Waiting for execution layer sync, Connected peers: 10
Feb 04 06:47:59 hoenir teku[452]: 06:47:59.991 WARN  - Request for eth1 deposit logs from 1 blocks failed. Retrying with a smaller block range.
Feb 04 06:48:01 hoenir teku[452]: 06:48:01.993 WARN  - Request for eth1 deposit logs from 1 blocks failed. Retrying with a smaller block range.
Feb 04 06:48:03 hoenir teku[452]: 06:48:03.995 WARN  - Request for eth1 deposit logs from 1 blocks failed. Retrying with a smaller block range.
Feb 04 06:48:05 hoenir teku[452]: 06:48:05.997 WARN  - Request for eth1 deposit logs from 1 blocks failed. Retrying with a smaller block range.
Feb 04 06:48:08 hoenir teku[452]: 06:48:08.000 WARN  - Request for eth1 deposit logs from 1 blocks failed. Retrying with a smaller block range.
Feb 04 06:48:10 hoenir teku[452]: 06:48:10.002 WARN  - Request for eth1 deposit logs from 1 blocks failed. Retrying with a smaller block range.
smasher816 commented 1 year ago

After a full resync I am now hitting a different issue. https://github.com/NethermindEth/nethermind/issues/5235