search

hermeznetwork / hermez-node

Hermez node Go implementation
GNU Affero General Public License v3.0
58 stars 33 forks source link

Improve Sync process to handle blocks with SC Events #1113

Closed tclemos closed 3 years ago

tclemos commented 3 years ago

What does this PR does?

Replaces the selection of the next block to sync by a logic that will look for the next block containing events in the SC instead of appending +1.

How to test?

start a brand new node and sync it

tclemos commented 3 years ago

Integration tests evidence! 

Starting test
    ✓ Init variables (4395ms)
    ✓ Check ETH L1 deposit (58301ms)
    ✓ Check ERC20 L1 deposit (44320ms)
    ✓ Check ETH L1 force exit (38253ms)
    ✓ Check ERC20 L1 force exit (48269ms)
    ✓ Check ETH withdrawal from firsts forceExits (16243ms)
    ✓ Check ERC20 withdrawal from firsts forceExits (12198ms)
    ✓ Check ETH N L1 force exits (34228ms)
    ✓ Check ETH withdrawal from previous N force exits (8158ms)
    ✓ Check single ETH L2 transfer (40337ms)
    ✓ Check single ERC20 L2 transfer (40347ms)
    ✓ Transfer ETH to a non-existent Bjj address (50334ms)
    ✓ Transfer ERC20 to a non-existent Bjj address (40261ms)
    ✓ Transfer ETH to hermez ethereum address (40251ms)
    ✓ Transfer ERC20 to hermez ethereum address (20207ms)
    ✓ Transfer ETH to non-existent ETH address (30252ms)
    ✓ Transfer ETH to hermez ethereum address from internal account (40281ms)
    ✓ Transfer ERC20 to hermez ethereum address from internal account (40272ms)
    ✓ Check multiple L2 ETH transfer in same batch (30797ms)
    ✓ Check multiple L2 ETH transfer in different batches (161272ms)
    ✓ Atomic txs (40437ms)
    ✓ Check L2 ETH exit (40247ms)
    ✓ Check L2 ERC20 exit (50331ms)
    ✓ Check ETH withdrawal from L2 exit (4097ms)
    ✓ Check ERC20 withdrawal from L2 exit (8116ms)
    ✓ Exit ETH and circuit withdrawal (43105ms)
    ✓ Assert API endpoints (1956ms)
    ✓ Update Buckets (12239ms)
    ✓ Check ETH L1 force exit bucket (34245ms)
    ✓ Check ERC20 L1 force exit bucket (48382ms)
    ✓ Check ETH withdrawal buckets (16209ms)
    ✓ Check ERC20 withdrawal buckets (12196ms)
    ✓ Check withdrawals buckets

  33 passing (19m)

Thanks you @marioidival for the help with the integration tests.

tclemos commented 3 years ago

Guys, I've found a bug with the sync reorg situation.

Basically, the node was counting on having all the blocks in the database in order to revert the synchronized data.

Integration tests are running in order to guarantee it does not break anything else, I'll post the results when it's done.

Since we didn't get this during the first run of the integration tests, I think we are missing a reorg integration test. @krlosMata can you help us with it?

@jeffprestes and @marioidival could you please review this PR once again? sorry for that, even with a lot of unit, integration, and manual tests, I couldn't get this error before, luckily the network provided this for us in the real environment before merging it to the next release.

tclemos commented 3 years ago

so far, so good!

web3-shh package will be deprecated in version 1.3.5 and will no longer be supported.
web3-bzz package will be deprecated in version 1.3.5 and will no longer be supported.

  E2E: HermezJS basic flow
Nothing to compile
executing MakeFile to initalize all the services in dockers
Starting test
    ✓ Init variables (8491ms)
    ✓ Check ETH L1 deposit (58668ms)
    ✓ Check ERC20 L1 deposit (44939ms)
    ✓ Check ETH L1 force exit (44324ms)
    ✓ Check ERC20 L1 force exit (48352ms)
    ✓ Check ETH withdrawal from firsts forceExits (16338ms)
    ✓ Check ERC20 withdrawal from firsts forceExits (12298ms)
    ✓ Check ETH N L1 force exits (38374ms)
    ✓ Check ETH withdrawal from previous N force exits (4117ms)
    ✓ Check single ETH L2 transfer (40458ms)
    ✓ Check single ERC20 L2 transfer (40382ms)
    ✓ Transfer ETH to a non-existent Bjj address (40371ms)
    ✓ Transfer ERC20 to a non-existent Bjj address (30338ms)
    ✓ Transfer ETH to hermez ethereum address (50433ms)
    ✓ Transfer ERC20 to hermez ethereum address (40513ms)
    ✓ Transfer ETH to non-existent ETH address (40344ms)
    ✓ Transfer ETH to hermez ethereum address from internal account (50507ms)
    ✓ Transfer ERC20 to hermez ethereum address from internal account (40400ms)
    ✓ Check multiple L2 ETH transfer in same batch (42353ms)
    ✓ Check multiple L2 ETH transfer in different batches (171641ms)
    ✓ Atomic txs (50526ms)
    ✓ Check L2 ETH exit (40344ms)
    ✓ Check L2 ERC20 exit (20311ms)
    ✓ Check ETH withdrawal from L2 exit (8194ms)
    ✓ Check ERC20 withdrawal from L2 exit (8230ms)
    ✓ Exit ETH and circuit withdrawal (40716ms)
    ✓ Assert API endpoints (3285ms)
    ✓ Update Buckets (16465ms)
    ✓ Check ETH L1 force exit bucket (34363ms)
    ✓ Check ERC20 L1 force exit bucket (48374ms)
    ✓ Check ETH withdrawal buckets (16435ms)
    ✓ Check ERC20 withdrawal buckets (12494ms)
    ✓ Check withdrawals buckets

  33 passing (20m)