blast-io / blast

97 stars 53 forks source link

Ecotone - Blast Geth exits with `panic: runtime error: invalid memory address or nil pointer dereference` #12

Open 8times4 opened 6 months ago

8times4 commented 6 months ago

Hey Team,

Since the ecotone update, all of our blast nodes exit with a panic. We're running it as binary, not via docker.

tried doing geth-init with the new genesis.json from deployment repo, but despite that being successful, our node still fails to run.

Full logs:

INFO [05-28|12:03:28.195] Starting Geth on Ethereum mainnet... 
INFO [05-28|12:03:28.195] Bumping default cache on mainnet         provided=1024 updated=4096
INFO [05-28|12:03:28.195] Enabling metrics collection 
INFO [05-28|12:03:28.195] Enabling stand-alone metrics HTTP endpoint address=10.0.0.10:6061
INFO [05-28|12:03:28.195] Starting metrics server                  addr=http://10.0.0.10:6061/debug/metrics
INFO [05-28|12:03:28.198] Maximum peer count                       ETH=0 LES=0 total=0
INFO [05-28|12:03:28.199] Smartcard socket not found, disabling    err="stat /run/pcscd/pcscd.comm: no such file or directory"
INFO [05-28|12:03:28.203] Enabling recording of key preimages since archive mode is used 
WARN [05-28|12:03:28.203] Disabled transaction unindexing for archive node 
INFO [05-28|12:03:28.203] Set global gas cap                       cap=50,000,000
INFO [05-28|12:03:28.203] Initializing the KZG library             backend=gokzg
INFO [05-28|12:03:28.240] Allocated trie memory caches             clean=1.20GiB dirty=0.00B
INFO [05-28|12:03:28.757] Using pebble as the backing database 
INFO [05-28|12:03:28.757] Allocated cache and file handles         database=/snapshot/blast-data/geth/geth/chaindata cache=2.00GiB handles=524,288
INFO [05-28|12:03:34.320] Opened ancient database                  database=/snapshot/blast-data/geth/geth/chaindata/ancient/chain readonly=false
INFO [05-28|12:03:34.322] State scheme set to already existing     scheme=hash
INFO [05-28|12:03:35.773] Initialising Ethereum protocol           network=81457 dbversion=8
INFO [05-28|12:03:35.773]  
INFO [05-28|12:03:35.773] --------------------------------------------------------------------------------------------------------------------------------------------------------- 
INFO [05-28|12:03:35.773] Chain ID:  81457 (unknown) 
INFO [05-28|12:03:35.773] Consensus: Optimism 
INFO [05-28|12:03:35.773]  
INFO [05-28|12:03:35.773] Pre-Merge hard forks (block based): 
INFO [05-28|12:03:35.773]  - Homestead:                   #0        (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/homestead.md) 
INFO [05-28|12:03:35.773]  - Tangerine Whistle (EIP 150): #0        (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/tangerine-whistle.md) 
INFO [05-28|12:03:35.773]  - Spurious Dragon/1 (EIP 155): #0        (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/spurious-dragon.md) 
INFO [05-28|12:03:35.773]  - Spurious Dragon/2 (EIP 158): #0        (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/spurious-dragon.md) 
INFO [05-28|12:03:35.773]  - Byzantium:                   #0        (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/byzantium.md) 
INFO [05-28|12:03:35.773]  - Constantinople:              #0        (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/constantinople.md) 
INFO [05-28|12:03:35.773]  - Petersburg:                  #0        (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/petersburg.md) 
INFO [05-28|12:03:35.773]  - Istanbul:                    #0        (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/istanbul.md) 
INFO [05-28|12:03:35.773]  - Muir Glacier:                #0        (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/muir-glacier.md) 
INFO [05-28|12:03:35.773]  - Berlin:                      #0        (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/berlin.md) 
INFO [05-28|12:03:35.773]  - London:                      #0        (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/london.md) 
INFO [05-28|12:03:35.773]  - Arrow Glacier:               #0        (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/arrow-glacier.md) 
INFO [05-28|12:03:35.773]  - Gray Glacier:                #0        (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/gray-glacier.md) 
INFO [05-28|12:03:35.773]  
INFO [05-28|12:03:35.773] Merge configured: 
INFO [05-28|12:03:35.773]  - Hard-fork specification:    https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/paris.md 
INFO [05-28|12:03:35.773]  - Network known to be merged: true 
INFO [05-28|12:03:35.773]  - Total terminal difficulty:  0 
INFO [05-28|12:03:35.773]  - Merge netsplit block:       #0        
INFO [05-28|12:03:35.773]  
INFO [05-28|12:03:35.773] Post-Merge hard forks (timestamp based): 
INFO [05-28|12:03:35.773]  - Shanghai:                    @0          (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/shanghai.md) 
INFO [05-28|12:03:35.773]  - Cancun:                      @1716843599 
INFO [05-28|12:03:35.773]  - Regolith:                    @0          
INFO [05-28|12:03:35.773]  - Canyon:                      @0          
INFO [05-28|12:03:35.773]  - Ecotone:                     @1716843599 
INFO [05-28|12:03:35.773]  
INFO [05-28|12:03:35.773] --------------------------------------------------------------------------------------------------------------------------------------------------------- 
INFO [05-28|12:03:35.773]  
INFO [05-28|12:03:35.774] Loaded most recent local block           number=4,016,892 hash=d2c457..0ab0cf td=0 age=15h3m36s
INFO [05-28|12:03:35.774] Loaded most recent local finalized block number=4,016,892 hash=d2c457..0ab0cf td=0 age=15h3m36s
INFO [05-28|12:03:35.808] Initialising Ethereum protocol           network=81457 dbversion=8
INFO [05-28|12:03:35.808] Initialized transaction indexer          limit=0
INFO [05-28|12:03:35.809] Loaded local transaction journal         transactions=0 dropped=0
INFO [05-28|12:03:35.809] Regenerated local transaction journal    transactions=0 accounts=0
INFO [05-28|12:03:35.809] Chain post-merge, sync via beacon client 
INFO [05-28|12:03:35.809] Gasprice oracle is ignoring threshold set threshold=2
WARN [05-28|12:03:35.812] Old unclean shutdowns found              count=154
WARN [05-28|12:03:35.812] Unclean shutdown detected                booted=2024-05-28T10:15:27+0000 age=1h48m8s
WARN [05-28|12:03:35.812] Unclean shutdown detected                booted=2024-05-28T10:15:37+0000 age=1h47m58s
WARN [05-28|12:03:35.812] Unclean shutdown detected                booted=2024-05-28T10:15:59+0000 age=1h47m36s
WARN [05-28|12:03:35.812] Unclean shutdown detected                booted=2024-05-28T10:17:41+0000 age=1h45m54s
WARN [05-28|12:03:35.812] Unclean shutdown detected                booted=2024-05-28T10:22:52+0000 age=1h40m43s
WARN [05-28|12:03:35.812] Unclean shutdown detected                booted=2024-05-28T10:25:06+0000 age=1h38m29s
WARN [05-28|12:03:35.812] Unclean shutdown detected                booted=2024-05-28T11:08:56+0000 age=54m39s
WARN [05-28|12:03:35.812] Unclean shutdown detected                booted=2024-05-28T11:29:15+0000 age=34m20s
WARN [05-28|12:03:35.812] Unclean shutdown detected                booted=2024-05-28T11:34:55+0000 age=28m40s
WARN [05-28|12:03:35.812] Unclean shutdown detected                booted=2024-05-28T11:35:29+0000 age=28m6s
WARN [05-28|12:03:35.812] Unclean shutdown detected                booted=2024-05-28T11:38:06+0000 age=25m29s
WARN [05-28|12:03:35.813] Engine API enabled                       protocol=eth
INFO [05-28|12:03:35.813] Starting peer-to-peer node               instance=Geth/v0.1.0-unstable-706acdde-20240521/linux-arm64/go1.22.2
INFO [05-28|12:03:35.826] IPC endpoint opened                      url=/snapshot/blast-data/geth/geth.ipc
INFO [05-28|12:03:35.826] Loaded JWT secret file                   path=/secrets/blast-jwt.hex crc32=0x5c2b8cd0
INFO [05-28|12:03:35.827] HTTP server started                      endpoint=10.0.0.10:8640 auth=false prefix= cors=* vhosts=localhost
INFO [05-28|12:03:35.827] New local node record                    seq=1,714,167,187,005 id=f09cf788294c54e1 ip=127.0.0.1 udp=0 tcp=30304
INFO [05-28|12:03:35.827] WebSocket enabled                        url=ws://10.0.0.10:8641
INFO [05-28|12:03:35.827] Started P2P networking                   self="enode://<redacted>@127.0.0.1:30304?discport=0"
INFO [05-28|12:03:35.827] WebSocket enabled                        url=ws://127.0.0.1:8651
INFO [05-28|12:03:35.827] HTTP server started                      endpoint=127.0.0.1:8651 auth=true  prefix= cors=localhost vhosts=localhost
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xe8c950]

goroutine 4381 [running]:
github.com/ethereum/go-ethereum/miner.(*worker).prepareWork(0x400e654908, 0x40157dd740)
    github.com/ethereum/go-ethereum/miner/worker.go:1038 +0x5c0
github.com/ethereum/go-ethereum/miner.(*worker).generateWork(0x400e654908, 0x40157dd740)
    github.com/ethereum/go-ethereum/miner/worker.go:1101 +0x38
github.com/ethereum/go-ethereum/miner.(*worker).mainLoop(0x400e654908)
    github.com/ethereum/go-ethereum/miner/worker.go:557 +0x2f0
created by github.com/ethereum/go-ethereum/miner.newWorker in goroutine 1
    github.com/ethereum/go-ethereum/miner/worker.go:295 +0x640

Thanks, 8x4