search

skalenetwork / skaled

Running more than 20 production blockchains, SKALED is Ethereum-compatible, high performance C++ Proof-of-Stake client, tools and libraries. Uses SKALE consensus as a blockchain consensus core. Includes dynamic Oracle. Implements file storage and retrieval as an EVM extension.
https://skale.network
GNU General Public License v3.0
84 stars 39 forks source link

Sync node doesn't broadcasts transactions when moved them from future tx queue into current tx queue #1774

Open oleksandrSydorenkoJ opened 9 months ago

oleksandrSydorenkoJ commented 9 months ago

Describe the bug Skaled on the sync node moved transactions from the Future transaction queue into the current transaction queue with category 0 and assigned category 1 for broadcasted transactions. But in a high amount of future transactions, some of them aren't broadcasted to core nodes.

Preconditions Active schains with enabled MTM and connected sync-node Address A with Sfuel on the balance

Versions 3.18.0-develop.18

To Reproduce

  1. Check nonce for Address A
  2. Send Send 150 future transactions with nonces: current nonce +1, current noce+2,...current nonce + 150
  3. Check all 150 transactions exist in a future transaction queue on the sync node
  4. send 1 transaction with the current nonce
  5. check the transaction queue

Expected behavior All transactions from the sync node should be broadcasted to core nodes and mined

Actual state Skaled has broadcasted only 12-15 transactions, that have been mined with success. Other 138 transactions are stucks in a current transaction queue, core nodes haven't received these transactions. 

### Tasks
PolinaKiporenko commented 7 months ago

Problem exist on core nodes also

PolinaKiporenko commented 7 months ago

moved to 2.4 release

PolinaKiporenko commented 4 months ago

3.19.0-beta.0

oleksandrSydorenkoJ commented 4 months ago

Verified on regression network skale_schain_carefree-knowing-sarin skalenetwork/schain:3.19.0-beta.1

727+272 = 999 txs

2024-05-31 15:49:14.595411   Block 129756 DB usage is 1113. Piece DB usage is 1457309 bytes
2024-05-31 15:49:14.595756   Successfully imported 0 of 0 transactions
2024-05-31 15:49:14.595776   TQBYTES:CTQ:0:FTQ:105507:TQSIZE:CTQ:0:FTQ:998

2024-05-31 15:49:15.152073   http://77.253.252.150:35964 >>> {"id":15,"jsonrpc":"2.0","method":"eth_sendRawTransaction","params":["0xf8663b830186a08252089488fd5e01078629cc194c933d9631b9448fe10b1d..."]}
2024-05-31 15:49:15.191411   Queued vaguely legit-looking transaction #3e7bfa84…
2024-05-31 15:49:15.191574   http://77.253.252.150:35964 <<< {"id":15,"jsonrpc":"2.0","result":"0x3e7bfa84adf4bc8c7f0b7ca57c8207f3a0179958215ba2f7989ff333ad1ea156"}

[2024-05-31 15:49:23.295] [0:main] [info] 129756:BLOCK_COMMITED: PRPSR:5:BID: 129757:ROOT:44553596519564103679452973953443314054560440155475749026936343393695283828026:HASH:1696b644:BLOCK_TXS:727:DMSG:0:TPRPS:17:MPRPS:1:RPRPS:15:TXS:16384:TXLS:16:MGS:99:INSTS:322:BPS:0:HDRS:5:SOCK:0:FDS:418:PRT:1689:BTA:7753:BSA:0:TPS:0:LWT:0:LRT:0:LWC:1133978:LRC:1553228:KNWN:16384:CONS:0:DSDS:0:SET:76:SBT:7:SEC:9854:SBC:28215:ZSC:1:EPT:1:STAMP:1717170555.696

...
[2024-05-31 15:49:24.706] [0:main] [info] 129757:BLOCK_COMMITED: PRPSR:7:BID: 129758:ROOT:44553596519564103679452973953443314054560440155475749026936343393695283828026:HASH:a68469f9:BLOCK_TXS:272:DMSG:68:TPRPS:17:MPRPS:1:RPRPS:15:TXS:16656:TXLS:16:MGS:189:INSTS:329:BPS:0:HDRS:101:SOCK:0:FDS:418:PRT:601:BTA:7753:BSA:0:TPS:0:LWT:0:LRT:0:LWC:1134069:LRC:1553353:KNWN:16384:CONS:0:DSDS:0:SET:76:SBT:7:SEC:9855:SBC:28216:ZSC:1:EPT:571:STAMP:1717170563.830

2024-05-31 15:49:24.809348   Block sealed #129758 (#c3f7c294…)
2024-05-31 15:49:24.811519   SWT:97:BFT:939
2024-05-31 15:49:24.811526   Successfully imported 272 of 272 transactions
2024-05-31 15:49:24.811532   TQBYTES:CTQ:0:FTQ:0:TQSIZE:CTQ:0:FTQ:0
oleksandrSydorenkoJ commented 3 months ago

skaled Core node skalenetwork/schain:3.19.0-beta.1-hwm-limts-release skale_schain_carefree-knowing-sarin send 150 transactions 

2024-06-12 12:36:35.705754   TQBYTES:CTQ:0:FTQ:0:TQSIZE:CTQ:0:FTQ:0
2024-06-12 12:36:46.675342   TQBYTES:CTQ:0:FTQ:5241:TQSIZE:CTQ:0:FTQ:49
2024-06-12 12:36:57.618317   TQBYTES:CTQ:0:FTQ:5241:TQSIZE:CTQ:0:FTQ:49
2024-06-12 12:37:08.368685   TQBYTES:CTQ:0:FTQ:15940:TQSIZE:CTQ:0:FTQ:149
2024-06-12 12:37:19.277827   TQBYTES:CTQ:0:FTQ:15940:TQSIZE:CTQ:0:FTQ:149
2024-06-12 12:37:30.211538   TQBYTES:CTQ:0:FTQ:15940:TQSIZE:CTQ:0:FTQ:149
2024-06-12 12:37:41.120642   TQBYTES:CTQ:0:FTQ:15940:TQSIZE:CTQ:0:FTQ:149
2024-06-12 12:37:52.059474   TQBYTES:CTQ:0:FTQ:15940:TQSIZE:CTQ:0:FTQ:149
2024-06-12 12:38:05.682434   TQBYTES:CTQ:0:FTQ:15940:TQSIZE:CTQ:0:FTQ:149
2024-06-12 12:38:17.401050   TQBYTES:CTQ:0:FTQ:15940:TQSIZE:CTQ:0:FTQ:149
2024-06-12 12:38:24.408840   TQBYTES:CTQ:0:FTQ:15940:TQSIZE:CTQ:0:FTQ:149
2024-06-12 12:38:32.243346   TQBYTES:CTQ:14763:FTQ:0:TQSIZE:CTQ:138:FTQ:0
2024-06-12 12:38:42.673403   TQBYTES:CTQ:14763:FTQ:0:TQSIZE:CTQ:138:FTQ:0
2024-06-12 12:38:53.192945   TQBYTES:CTQ:14763:FTQ:0:TQSIZE:CTQ:138:FTQ:0
2024-06-12 12:39:04.082685   TQBYTES:CTQ:0:FTQ:0:TQSIZE:CTQ:0:FTQ:0

send 1000 transactions (5 batches 200 transactions)

2024-06-12 12:39:15.515605   TQBYTES:CTQ:0:FTQ:21290:TQSIZE:CTQ:0:FTQ:199
2024-06-12 12:39:26.389541   TQBYTES:CTQ:0:FTQ:21290:TQSIZE:CTQ:0:FTQ:199
2024-06-12 12:39:37.014142   TQBYTES:CTQ:0:FTQ:21290:TQSIZE:CTQ:0:FTQ:199
2024-06-12 12:39:47.929402   TQBYTES:CTQ:0:FTQ:21290:TQSIZE:CTQ:0:FTQ:199
2024-06-12 12:39:58.905921   TQBYTES:CTQ:0:FTQ:42689:TQSIZE:CTQ:0:FTQ:399
2024-06-12 12:40:09.853054   TQBYTES:CTQ:0:FTQ:64086:TQSIZE:CTQ:0:FTQ:599
2024-06-12 12:40:20.831547   TQBYTES:CTQ:0:FTQ:85482:TQSIZE:CTQ:0:FTQ:799
2024-06-12 12:40:31.376273   TQBYTES:CTQ:0:FTQ:106880:TQSIZE:CTQ:0:FTQ:999
2024-06-12 12:40:42.306535   TQBYTES:CTQ:0:FTQ:106880:TQSIZE:CTQ:0:FTQ:999
2024-06-12 12:40:43.975341   TQBYTES:CTQ:106880:FTQ:0:TQSIZE:CTQ:999:FTQ:0
2024-06-12 12:40:45.024660   TQBYTES:CTQ:105918:FTQ:0:TQSIZE:CTQ:990:FTQ:0
2024-06-12 12:40:55.459753   TQBYTES:CTQ:105918:FTQ:0:TQSIZE:CTQ:990:FTQ:0
2024-06-12 12:41:06.359906   TQBYTES:CTQ:105918:FTQ:0:TQSIZE:CTQ:990:FTQ:0
2024-06-12 12:41:17.274990   TQBYTES:CTQ:105918:FTQ:0:TQSIZE:CTQ:990:FTQ:0
2024-06-12 12:41:27.751727   TQBYTES:CTQ:105918:FTQ:0:TQSIZE:CTQ:990:FTQ:0
2024-06-12 12:41:39.531823   TQBYTES:CTQ:0:FTQ:0:TQSIZE:CTQ:0:FTQ:0
2024-06-12 12:41:50.424146   TQBYTES:CTQ:0:FTQ:0:TQSIZE:CTQ:0:FTQ:0
2024-06-12 12:42:00.884569   TQBYTES:CTQ:0:FTQ:0:TQSIZE:CTQ:0:FTQ:0
olehnikolaiev commented 3 months ago

reverted changes https://github.com/skalenetwork/skaled/pull/1920

oleksandrSydorenkoJ commented 3 months ago

sync node skalenetwork/schain:3.19.0-beta.1-hwm-limts-release

03:04.055336   TQBYTES:CTQ:0:FTQ:0:TQSIZE:CTQ:0:FTQ:0
2024-06-12 19:03:14.174892   TQBYTES:CTQ:0:FTQ:0:TQSIZE:CTQ:0:FTQ:0
2024-06-12 19:03:29.297140   TQBYTES:CTQ:0:FTQ:18507:TQSIZE:CTQ:0:FTQ:173
2024-06-12 19:03:39.425665   TQBYTES:CTQ:0:FTQ:18507:TQSIZE:CTQ:0:FTQ:173
2024-06-12 19:03:49.551129   TQBYTES:CTQ:642:FTQ:18079:TQSIZE:CTQ:6:FTQ:169
2024-06-12 19:03:49.560980   TQBYTES:CTQ:642:FTQ:18079:TQSIZE:CTQ:6:FTQ:169
2024-06-12 19:03:54.680952   TQBYTES:CTQ:0:FTQ:18079:TQSIZE:CTQ:0:FTQ:169
2024-06-12 19:04:09.807408   TQBYTES:CTQ:1177:FTQ:17009:TQSIZE:CTQ:11:FTQ:159
2024-06-12 19:04:09.824279   TQBYTES:CTQ:1177:FTQ:17009:TQSIZE:CTQ:11:FTQ:159
2024-06-12 19:04:19.843885   TQBYTES:CTQ:107:FTQ:17009:TQSIZE:CTQ:1:FTQ:159