search

ethpandaops / dora

Dora the Explorer is a lightweight slot explorer for the ethereum beaconchain
https://beaconlight.ephemery.dev/
GNU General Public License v3.0
79 stars 26 forks source link

Fork detection takes too long to keep up with the chain during long unfinality #114

Closed pk910 closed 2 months ago

pk910 commented 2 months ago

The new fork detection takes too long when the network has large unfinality time. This leads to dora "lagging behind" the chain head with it's fork & canonical chain computation:

image (fork & canonical chain computation is 3 blocks behind)

pprof also shows subroutines being stuck in the fork detection code:

2 @ 0x443b2e 0x4569a5 0x456974 0x477b05 0x48725d 0x1010b95 0x1010b69 0xfffd91 0xffeed7 0xffe3b1 0xffdf85 0xffd4aa 0x47b9e1
#   0x477b04    sync.runtime_SemacquireMutex+0x24                       /opt/hostedtoolcache/go/1.22.6/x64/src/runtime/sema.go:77
#   0x48725c    sync.(*Mutex).lockSlow+0x15c                            /opt/hostedtoolcache/go/1.22.6/x64/src/sync/mutex.go:171
#   0x1010b94   sync.(*Mutex).Lock+0x74                             /opt/hostedtoolcache/go/1.22.6/x64/src/sync/mutex.go:90
#   0x1010b68   github.com/ethpandaops/dora/indexer/beacon.(*forkCache).processBlock+0x48   /home/runner/work/dora/dora/indexer/beacon/forkcache.go:305
#   0xfffd90    github.com/ethpandaops/dora/indexer/beacon.(*Client).processBlock+0x3b0     /home/runner/work/dora/dora/indexer/beacon/client.go:367
#   0xffeed6    github.com/ethpandaops/dora/indexer/beacon.(*Client).processStreamBlock+0x36    /home/runner/work/dora/dora/indexer/beacon/client.go:266
#   0xffe3b0    github.com/ethpandaops/dora/indexer/beacon.(*Client).processHeadEvent+0xf0  /home/runner/work/dora/dora/indexer/beacon/client.go:180
#   0xffdf84    github.com/ethpandaops/dora/indexer/beacon.(*Client).runClientLoop+0x824    /home/runner/work/dora/dora/indexer/beacon/client.go:153
#   0xffd4a9    github.com/ethpandaops/dora/indexer/beacon.(*Client).startClientLoop+0x69   /home/runner/work/dora/dora/indexer/beacon/client.go:99

1 @ 0x415e85 0x10103b7 0x1010e55 0xfffd91 0xffeed7 0xffe285 0xffdd97 0xffd4aa 0x47b9e1
#   0x10103b6   github.com/ethpandaops/dora/indexer/beacon.(*forkCache).checkForkDistance+0x156 /home/runner/work/dora/dora/indexer/beacon/forkcache.go:221
#   0x1010e54   github.com/ethpandaops/dora/indexer/beacon.(*forkCache).processBlock+0x334  /home/runner/work/dora/dora/indexer/beacon/forkcache.go:338
#   0xfffd90    github.com/ethpandaops/dora/indexer/beacon.(*Client).processBlock+0x3b0     /home/runner/work/dora/dora/indexer/beacon/client.go:367
#   0xffeed6    github.com/ethpandaops/dora/indexer/beacon.(*Client).processStreamBlock+0x36    /home/runner/work/dora/dora/indexer/beacon/client.go:266
#   0xffe284    github.com/ethpandaops/dora/indexer/beacon.(*Client).processBlockEvent+0xc4 /home/runner/work/dora/dora/indexer/beacon/client.go:169
#   0xffdd96    github.com/ethpandaops/dora/indexer/beacon.(*Client).runClientLoop+0x636    /home/runner/work/dora/dora/indexer/beacon/client.go:148
#   0xffd4a9    github.com/ethpandaops/dora/indexer/beacon.(*Client).startClientLoop+0x69   /home/runner/work/dora/dora/indexer/beacon/client.go:99