With the introduction of PayloadTimeout, transactions no longer stop committing in many cases even when there are a large number of high-load processes.
On the other hand, different problems seem to exist in specific cases.
Premise: This occurs on networks with high BlockGasLimit (e.g. 800000000).
This occurs when there are many transactions that store large data. If a block contains many transactions, the time required to commit all transactions increases.
The commit process will be completed until the end, but the block will be judged as old block data (err="old message").
As a concrete example, the following situation occurs:
INFO [03-20|12:34:21.048] QBFT: changed state address=0x35D56A7515e824BE4122f033D60063D035573a0c current.round=0 current.sequence=2693 old.state=Committed new.state="Accept request"
INFO [03-20|12:34:21.048] QBFT: start new round address=0x35D56A7515e824BE4122f033D60063D035573a0c old.round=0 old.sequence=2692 old.state=Committed old.proposer=0x03Ee8c85944b16DFA517cB0DdeFe123c7341A534 next.round=0 next.seq=2693 next.proposer=0x35D56A7515e824BE4122f033D60063D035573a0c next.valSet="[0x03Ee8c85944b16DFA517cB0DdeFe123c7341A534 0x35D56A7515e824BE4122f033D60063D035573a0c 0x47A847fbDF801154253593851aC9A2E775323534 0xc25d04978fd86ee604FeB88f3C635D555eB6D42D]" next.size=4 next.IsProposer=true
INFO [03-20|12:34:21.067] Aborting transaction processing signal=1
INFO [03-20|12:34:21.068] 🔗 block reached canonical chain number=2685 hash=9e0494..eba849
INFO [03-20|12:34:21.069] Commit new mining work number=2693 sealhash=c23554..bf9c6f uncles=0 txs=0 gas=0 fees=0 elapsed=1.756ms
INFO [03-20|12:34:21.901] Aborting transaction processing signal=3
INFO [03-20|12:34:22.000] QBFT: handle block proposal request address=0x35D56A7515e824BE4122f033D60063D035573a0c current.round=0 current.sequence=2693 state="Accept request"
INFO [03-20|12:34:22.000] QBFT: broadcast PRE-PREPARE message address=0x35D56A7515e824BE4122f033D60063D035573a0c current.round=0 current.sequence=2693 state="Accept request" msg.code=18 msg.source=0x35D56A7515e824BE4122f033D60063D035573a0c msg.round=0 msg.sequence=2693 block.number=2693 block.hash=0xc23554e8cd8c89e282aaba66ec52f39d1a3ac54f4d8a545e40baf497f9bf9c6f payload=0xf902ccf902c6f90280820a8580f90279f90274a05a10e670ccd425db5cb3d807decb20031e95346e23d11cf8653197173c0ff13da01dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d493479435d56a7515e824be4122f033d60063d035573a0ca01aee659021a4b3b886f27e583c7f21d6079627367439044dfb53465e3779c11ea056e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421a056e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421b901000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001820a85842faf0800808465fad7ceb87cf87aa0d883010a03846765746888676f312e32322e31856c696e757800000000000000f8549403ee8c85944b16dfa517cb0ddefe123c7341a5349435d56a7515e824be4122f033d60063d035573a0c9447a847fbdf801154253593851ac9a2e77532353494c25d04978fd86ee604feb88f3c635d555eb6d42dc080c0a063746963616c2062797a616e74696e65206661756c7420746f6c6572616e6365880000000000000000c0c0b84104ac577e2cdec395abbca66ac241e23778a902a4f76bbfbb2cc63fb9d19935280d8b56bbb7910f41a23beabca218c0f111a8a915e016ae3f15725f320b0f116b00c2c0c0
INFO [03-20|12:34:22.001] QBFT: handle PRE-PREPARE message address=0x35D56A7515e824BE4122f033D60063D035573a0c
INFO [03-20|12:34:22.001] QBFT: accepted PRE-PREPARE message address=0x35D56A7515e824BE4122f033D60063D035573a0c
INFO [03-20|12:34:22.001] QBFT: changed state address=0x35D56A7515e824BE4122f033D60063D035573a0c current.round=0 current.sequence=2693 old.state="Accept request" new.state=Preprepared
INFO [03-20|12:34:22.001] QBFT: broadcast PREPARE message address=0x35D56A7515e824BE4122f033D60063D035573a0c current.round=0 current.sequence=2693 state=Preprepared msg.code=19 msg.source=0x35D56A7515e824BE4122f033D60063D035573a0c msg.round=0 msg.sequence=2693 payload=0xf869e5820a8580a0c23554e8cd8c89e282aaba66ec52f39d1a3ac54f4d8a545e40baf497f9bf9c6fb841413ca3f6af85af0f96f356f541bc8b5e4fa67226eadba5aada185be0bd9cc8647ad6e6137d3dc96ca220ae958a860b16bd5b288e334996dc6645c65b10fbc04700
INFO [03-20|12:34:22.001] QBFT: handle PREPARE message address=0x35D56A7515e824BE4122f033D60063D035573a0c current.round=0 current.sequence=2693 state=Preprepared msg.code=19 msg.source=0x35D56A7515e824BE4122f033D60063D035573a0c msg.round=0 msg.sequence=2693 prepares.count=0 quorum=3
INFO [03-20|12:34:22.003] QBFT: handle PREPARE message address=0x35D56A7515e824BE4122f033D60063D035573a0c current.round=0 current.sequence=2693 state=Preprepared msg.code=19 msg.source=0xc25d04978fd86ee604FeB88f3C635D555eB6D42D msg.round=0 msg.sequence=2693 prepares.count=1 quorum=3
INFO [03-20|12:34:22.004] QBFT: handle PREPARE message address=0x35D56A7515e824BE4122f033D60063D035573a0c current.round=0 current.sequence=2693 state=Preprepared msg.code=19 msg.source=0x03Ee8c85944b16DFA517cB0DdeFe123c7341A534 msg.round=0 msg.sequence=2693 prepares.count=2 quorum=3
INFO [03-20|12:34:22.004] QBFT: received quorum of PREPARE messages address=0x35D56A7515e824BE4122f033D60063D035573a0c current.round=0 current.sequence=2693 state=Preprepared msg.code=19 msg.source=0x03Ee8c85944b16DFA517cB0DdeFe123c7341A534 msg.round=0 msg.sequence=2693 prepares.count=3 quorum=3
INFO [03-20|12:34:22.004] QBFT: changed state address=0x35D56A7515e824BE4122f033D60063D035573a0c current.round=0 current.sequence=2693 old.state=Preprepared new.state=Prepared
INFO [03-20|12:34:22.004] QBFT: broadcast COMMIT message address=0x35D56A7515e824BE4122f033D60063D035573a0c current.round=0 current.sequence=2693 state=Prepared msg.code=20 msg.source=0x35D56A7515e824BE4122f033D60063D035573a0c msg.round=0 msg.sequence=2693 payload=0xf8adf868820a8580a0c23554e8cd8c89e282aaba66ec52f39d1a3ac54f4d8a545e40baf497f9bf9c6fb841323835c0c298a066814ead59882d1beda036e308aff4e1431d8464cf1a4586357ce16d838c9ad144ba188747ff37a12aa87d0be806b9549140114df4aa9b2af501b84173e0a1809db84d81a0c71a837b7b373e92bc308059de6fcf58a1f44e758ddba026599893b272e34fbfa47b28e8f3f80789f6927abe8002eed9ee17479cf0cc5901
INFO [03-20|12:34:22.006] QBFT: handle COMMIT message address=0x35D56A7515e824BE4122f033D60063D035573a0c current.round=0 current.sequence=2693 state=Prepared msg.code=20 msg.source=0x03Ee8c85944b16DFA517cB0DdeFe123c7341A534 msg.round=0 msg.sequence=2693 commits.count=0 quorum=3
INFO [03-20|12:34:22.007] QBFT: handle COMMIT message address=0x35D56A7515e824BE4122f033D60063D035573a0c current.round=0 current.sequence=2693 state=Prepared msg.code=20 msg.source=0x35D56A7515e824BE4122f033D60063D035573a0c msg.round=0 msg.sequence=2693 commits.count=1 quorum=3
INFO [03-20|12:34:22.009] QBFT: handle COMMIT message address=0x35D56A7515e824BE4122f033D60063D035573a0c current.round=0 current.sequence=2693 state=Prepared msg.code=20 msg.source=0x47A847fbDF801154253593851aC9A2E775323534 msg.round=0 msg.sequence=2693 commits.count=2 quorum=3
INFO [03-20|12:34:22.009] QBFT: received quorum of COMMIT messages address=0x35D56A7515e824BE4122f033D60063D035573a0c current.round=0 current.sequence=2693 state=Prepared msg.code=20 msg.source=0x47A847fbDF801154253593851aC9A2E775323534 msg.round=0 msg.sequence=2693 commits.count=3 quorum=3
INFO [03-20|12:34:22.009] QBFT: changed state address=0x35D56A7515e824BE4122f033D60063D035573a0c current.round=0 current.sequence=2693 old.state=Prepared new.state=Committed
INFO [03-20|12:34:22.009] BFT: block proposal committed author=0x35D56A7515e824BE4122f033D60063D035573a0c hash=c23554..bf9c6f number=2693
INFO [03-20|12:34:22.016] QBFT: handle final committed address=0x35D56A7515e824BE4122f033D60063D035573a0c current.round=0 current.sequence=2693 state=Committed
INFO [03-20|12:34:22.016] QBFT: initialize new round address=0x35D56A7515e824BE4122f033D60063D035573a0c current.round=0 current.sequence=2693 target.round=0 lastProposal.number=2693 lastProposal.hash=c23554..bf9c6f
INFO [03-20|12:34:22.017] QBFT: changed state address=0x35D56A7515e824BE4122f033D60063D035573a0c current.round=0 current.sequence=2694 old.state=Committed new.state="Accept request"
INFO [03-20|12:34:22.017] QBFT: start new round address=0x35D56A7515e824BE4122f033D60063D035573a0c old.round=0 old.sequence=2693 old.state=Committed old.proposer=0x35D56A7515e824BE4122f033D60063D035573a0c next.round=0 next.seq=2694 next.proposer=0x47A847fbDF801154253593851aC9A2E775323534 next.valSet="[0x03Ee8c85944b16DFA517cB0DdeFe123c7341A534 0x35D56A7515e824BE4122f033D60063D035573a0c 0x47A847fbDF801154253593851aC9A2E775323534 0xc25d04978fd86ee604FeB88f3C635D555eB6D42D]" next.size=4 next.IsProposer=false
INFO [03-20|12:34:22.017] Successfully sealed new block number=2693 sealhash=c23554..bf9c6f hash=c23554..bf9c6f elapsed=948.231ms
INFO [03-20|12:34:22.018] 🔨 mined potential block number=2693 hash=c23554..bf9c6f
INFO [03-20|12:34:23.002] QBFT: handle PRE-PREPARE message address=0x35D56A7515e824BE4122f033D60063D035573a0c
INFO [03-20|12:34:23.002] QBFT: accepted PRE-PREPARE message address=0x35D56A7515e824BE4122f033D60063D035573a0c
INFO [03-20|12:34:23.002] QBFT: changed state address=0x35D56A7515e824BE4122f033D60063D035573a0c current.round=0 current.sequence=2694 old.state="Accept request" new.state=Preprepared
INFO [03-20|12:34:23.002] QBFT: broadcast PREPARE message address=0x35D56A7515e824BE4122f033D60063D035573a0c current.round=0 current.sequence=2694 state=Preprepared msg.code=19 msg.source=0x35D56A7515e824BE4122f033D60063D035573a0c msg.round=0 msg.sequence=2694 payload=0xf869e5820a8680a083208a92a66aa710f2cb1d059fba3ebc7ac12fff95a38d57167ddab21d5e29d1b841c6b75958b5f21be308292e73abdd03e6559f12d434b900eb67fbcfbd77fba63c6d0811c1409d8969359b0828635ac6f01729b4a038051f7773dea93b2d723f1900
INFO [03-20|12:34:23.003] QBFT: handle PREPARE message address=0x35D56A7515e824BE4122f033D60063D035573a0c current.round=0 current.sequence=2694 state=Preprepared msg.code=19 msg.source=0x47A847fbDF801154253593851aC9A2E775323534 msg.round=0 msg.sequence=2694 prepares.count=0 quorum=3
INFO [03-20|12:34:23.003] QBFT: handle PREPARE message address=0x35D56A7515e824BE4122f033D60063D035573a0c current.round=0 current.sequence=2694 state=Preprepared msg.code=19 msg.source=0x35D56A7515e824BE4122f033D60063D035573a0c msg.round=0 msg.sequence=2694 prepares.count=1 quorum=3
INFO [03-20|12:34:23.006] QBFT: handle PREPARE message address=0x35D56A7515e824BE4122f033D60063D035573a0c current.round=0 current.sequence=2694 state=Preprepared msg.code=19 msg.source=0x03Ee8c85944b16DFA517cB0DdeFe123c7341A534 msg.round=0 msg.sequence=2694 prepares.count=2 quorum=3
INFO [03-20|12:34:23.006] QBFT: received quorum of PREPARE messages address=0x35D56A7515e824BE4122f033D60063D035573a0c current.round=0 current.sequence=2694 state=Preprepared msg.code=19 msg.source=0x03Ee8c85944b16DFA517cB0DdeFe123c7341A534 msg.round=0 msg.sequence=2694 prepares.count=3 quorum=3
INFO [03-20|12:34:23.006] QBFT: changed state address=0x35D56A7515e824BE4122f033D60063D035573a0c current.round=0 current.sequence=2694 old.state=Preprepared new.state=Prepared
INFO [03-20|12:34:23.006] QBFT: broadcast COMMIT message address=0x35D56A7515e824BE4122f033D60063D035573a0c current.round=0 current.sequence=2694 state=Prepared msg.code=20 msg.source=0x35D56A7515e824BE4122f033D60063D035573a0c msg.round=0 msg.sequence=2694 payload=0xf8adf868820a8680a083208a92a66aa710f2cb1d059fba3ebc7ac12fff95a38d57167ddab21d5e29d1b841e8157bf19f7ffeb2e242a8c99fc1b1de1d42270346ba2962e4f4e093f82f1c187b2f79959245b63f4f0906d840c8357696feb4baf55e82a29b36eaa1695ee47a01b8414dc2033f5ae2ad1c9b44385542a617052340abc7db777a5eab3681225ab80ef1712998285e1845650c0065282ca3e543c840def20dd6e646306e6813e117b4de00
INFO [03-20|12:34:23.009] QBFT: handle COMMIT message address=0x35D56A7515e824BE4122f033D60063D035573a0c current.round=0 current.sequence=2694 state=Prepared msg.code=20 msg.source=0x35D56A7515e824BE4122f033D60063D035573a0c msg.round=0 msg.sequence=2694 commits.count=0 quorum=3
INFO [03-20|12:34:23.014] QBFT: handle COMMIT message address=0x35D56A7515e824BE4122f033D60063D035573a0c current.round=0 current.sequence=2694 state=Prepared msg.code=20 msg.source=0x47A847fbDF801154253593851aC9A2E775323534 msg.round=0 msg.sequence=2694 commits.count=1 quorum=3
INFO [03-20|12:34:23.015] QBFT: handle COMMIT message address=0x35D56A7515e824BE4122f033D60063D035573a0c current.round=0 current.sequence=2694 state=Prepared msg.code=20 msg.source=0x03Ee8c85944b16DFA517cB0DdeFe123c7341A534 msg.round=0 msg.sequence=2694 commits.count=2 quorum=3
INFO [03-20|12:34:23.015] QBFT: received quorum of COMMIT messages address=0x35D56A7515e824BE4122f033D60063D035573a0c current.round=0 current.sequence=2694 state=Prepared msg.code=20 msg.source=0x03Ee8c85944b16DFA517cB0DdeFe123c7341A534 msg.round=0 msg.sequence=2694 commits.count=3 quorum=3
INFO [03-20|12:34:23.015] QBFT: changed state address=0x35D56A7515e824BE4122f033D60063D035573a0c current.round=0 current.sequence=2694 old.state=Prepared new.state=Committed
INFO [03-20|12:34:23.015] BFT: block proposal committed author=0x35D56A7515e824BE4122f033D60063D035573a0c hash=83208a..5e29d1 number=2694
❌ INFO [03-20|12:34:23.018] Commit new mining work number=2693 sealhash=96d99c..284c65 uncles=0 txs=37 gas=265,386,607 fees=0 elapsed=1.950s
INFO [03-20|12:34:23.019] QBFT: handle block proposal request address=0x35D56A7515e824BE4122f033D60063D035573a0c current.round=0 current.sequence=2694 state=Committed
❌ ERROR[03-20|12:34:23.019] QBFT: unexpected request address=0x35D56A7515e824BE4122f033D60063D035573a0c current.round=0 current.sequence=2694 state=Committed err="old message"
We may need to consider the "optimal" PayloadTimeout and BlockGasLimit values.
Other information
We tested with the following conditions.
Use a simple contract code like below.
Set the network BlockGasLimit to 80000000.
Send large string data (e.g. 10000 characters) in 1000 transactions at the same time.
pragma solidity ^0.8.0;
/// @title Contract for load testing
contract LoadTest {
struct CommitMessage {
uint256 blockNumber;
string message;
}
mapping(address => CommitMessage) public commit_messages;
constructor() {}
/// Store large string
function storeString(string memory _message) public {
CommitMessage storage commit_message = commit_messages[msg.sender];
commit_message.blockNumber = block.number;
commit_message.message = _message;
}
}
Describe the bug
With the introduction of PayloadTimeout, transactions no longer stop committing in many cases even when there are a large number of high-load processes.
On the other hand, different problems seem to exist in specific cases.
As a concrete example, the following situation occurs:
We may need to consider the "optimal" PayloadTimeout and BlockGasLimit values.
Other information
We tested with the following conditions.