search

ava-labs / avalanchego

Go implementation of an Avalanche node.
https://avax.network
BSD 3-Clause "New" or "Revised" License
2.1k stars 655 forks source link

bootstrap failure: block failed verification during bootstrapping due to invalid gas used (BAD BLOCK) #1122

Open vm-msardarov opened 1 year ago

vm-msardarov commented 1 year ago

We have had it several times at different blocks and restarted bootstrap from scratch several times already. And after these problems occur, the avalanche daemon shuts down, and after restart transactions in the rest of the blocks are not processed. When we call eth_getLogs API, the avalanchego daemon hangs on querying blocks with a block number higher than the failed one.

Questions:

  1. How can this error be fixed?
  2. How can we restart the blocks' processing (indexing) after "BAD BLOCK" to allow the eth_getLogs API to work?

Logs:

[08-21|14:43:41.520] INFO <C Chain> queue/jobs.go:174 executed 15325000 of 18422484 operations. ETA = 48h5m43s
ERROR[08-21|14:45:19.491] <C Chain> github.com/ava-labs/coreth/core/blockchain.go:1292:
########## BAD BLOCK #########
Chain config: Chain ID:  43114
Consensus: Dummy Consensus Engine

Hard Forks:
 - Homestead:                   0        (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/homestead.md)
 - DAO Fork:                    0        (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/dao-fork.md)
 - Tangerine Whistle (EIP 150): 0        (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/tangerine-whistle.md)
 - Spurious Dragon/1 (EIP 155): 0        (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/spurious-dragon.md)
 - Spurious Dragon/2 (EIP 158): 0        (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/spurious-dragon.md)
 - Byzantium:                   0        (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/byzantium.md)
 - Constantinople:              0        (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/constantinople.md)
 - Petersburg:                  0        (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/petersburg.md)
 - Istanbul:                    0        (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/istanbul.md)
 - Muir Glacier:                0        (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/muir-glacier.md)
 - Apricot Phase 1 Timestamp:   1617199200 (https://github.com/ava-labs/avalanchego/releases/tag/v1.3.0)
 - Apricot Phase 2 Timestamp:   1620644400 (https://github.com/ava-labs/avalanchego/releases/tag/v1.4.0)
 - Apricot Phase 3 Timestamp:   1629813600 (https://github.com/ava-labs/avalanchego/releases/tag/v1.5.0)
 - Apricot Phase 4 Timestamp:   1632344400 (https://github.com/ava-labs/avalanchego/releases/tag/v1.6.0)
 - Apricot Phase 5 Timestamp:   1638468000 (https://github.com/ava-labs/avalanchego/releases/tag/v1.7.0)

Number: 15327098
Hash: 0x47179648515aed708cf13822b146c6110a4918f8a648a998307050b2466af7d8
         0: cumulative: 464043 gas: 464043 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x7a90570388354717401072d8628a6f1cc16287a1741651952246498abdd7ec98 logs: [0xc1b2059080 0xc1b2059130 0xc1b20591e0] bloom: 00004000000000000000000000000008000000000000000000010000000004000000000000000000000000000000000000000000000000000000080000220000000000000000000000000008000000000000000080000000000000000000000000000000000000000400000000000000000000000004000000000010000000000000000000004000000000000000000000000000000100000000000000000000020000000000000000004008000000000000000000000000000000000000008000000002000000000000000000000000000000020000000000000000000000000010000000000000000000020000004000000000200000000000000000000000 state:
         1: cumulative: 508870 gas: 44827 contract: 0x0000000000000000000000000000000000000000 status: 0 tx: 0x45415fffdf932ecf93123a9aeb00046adf366f4b4ead685c05ab960ba2099e6f logs: [] bloom: 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 state:

Error: invalid gas used (remote: 735865 local: 508870)
##############################

[08-21|14:45:19.492] ERROR <C Chain> bootstrap/block_job.go:86 block WA3FKuD9Qzn3nviSFFvboTPhHoJEdbfiCKmHKPYoonRyiWzZ1 failed verification during bootstrapping due to invalid gas used (remote: 735865 local: 508870)
[08-21|14:45:19.496] FATAL <C Chain> handler/handler.go:276 shutting down chain due to unexpected error: failed to execute job WA3FKuD9Qzn3nviSFFvboTPhHoJEdbfiCKmHKPYoonRyiWzZ1 due to failed to verify block in bootstrapping: invalid gas used (remote: 735865 local: 508870) while processing sync message: (Op: ancestors, NodeID: NodeID-K2fyNZcrWA4DBUNyFme4p

To Reproduce:

Environment

aaxx commented 1 year ago

Also experienced a problem like this, couple of times in different blocks. Can you bump it?

github-actions[bot] commented 1 year ago

This issue has become stale because it has been open 60 days with no activity. Adding the lifecycle/frozen label will cause this issue to ignore lifecycle events.