search

skalenetwork / skaled

Running more than 20 production blockchains, SKALED is Ethereum-compatible, high performance C++ Proof-of-Stake client, tools and libraries. Uses SKALE consensus as a blockchain consensus core. Includes dynamic Oracle. Implements file storage and retrieval as an EVM extension.
https://skale.network
GNU General Public License v3.0
85 stars 39 forks source link

Skaled not starts from the catchup for a long time after recreation of skaled-container #1373

Open DmytroNazarenko opened 1 year ago

DmytroNazarenko commented 1 year ago

Versions Skaled: skalenetwork/schain:3.16.0-beta.6

Environment 20 active nodes on the network 8 schains up and running

Preconditions Load tests with blockchain-killer (1 GB of state in use)

Steps to Reproduce/Description

  1. Restart one of the nodes from the catchup from 0 block - 15 other nodes are active and under simple load 2-3 txs per block
  2. check sklaed logs

Expected behavior Skaled should start catching up blocks in 2-5 minutes after creation.

Actual result Skaled could not catchup blocks after 10-12 minutes of downtime and may be restarted by skale admin

skaled_not_catchup.log

DmytroNazarenko commented 3 weeks ago

Reproduced on mainnet: skale-node: 3.0.0 skaled: 3.19.0

dimalit commented 3 weeks ago

In the provided log, skaled shows first message related to catch-up after 9 minutes of work:

[2023-01-27 18:39:14.594] [14:catchup] [info] 0:CATCHUP_GOT_BLOCKS:COUNT:12968:STARTBLOCK:1

is this wrong? is there any limit on this time?