search

Consensys / quorum

A permissioned implementation of Ethereum supporting data privacy
https://www.goquorum.com/
GNU Lesser General Public License v3.0
4.68k stars 1.29k forks source link

IBFT fails to create new blocks #605

Closed prd-fox closed 5 years ago

prd-fox commented 5 years ago

System information

Geth version: 1.8.12

OS & Version: Linux

Branch, Commit Hash or Release: git status

Expected behaviour

Block making should continue at normal rate, even if transaction rate is higher

Actual behaviour

Block minting stops

Steps to reproduce the behaviour

Create an IBFT network with a high TPS (~600TPS) and wait for it to stall

A user on slack has reported that when a sustained high TPS occurs, block making stops and the transactions are queued in the pool. Sometimes it recovers, and other times it does not.

I have not been able to reproduce this behaviour locally, but my machine is not able to get up to the users TPS value (I can max out about 200).

The Istanbul block time is set to 1 second.

colindog commented 5 years ago

I also met this issue for some times. And not only on IBFT, RAFT also. I keep sending test transactions to some nodes. After some time, the pending queue of some node is stucked like:

txpool.status { pending: 10413, queued: 0 }

And this node stops dealing with new transactions. But the other nodes works well.

jbhurat commented 5 years ago

Hi @colindog, can you provide more information like Geth version, number of nodes, what is the rate at which you are executing transactions, are the transactions public or private etc.

colindog commented 5 years ago

Dear @jbhurat ， My geth version is as below: Geth Version: 1.8.12-stable Quorum Version: 2.2.1. I set a 7 nodes testnet. I only sent the transtractions to a single node. The TPS is aroud 100, all public. And I use the web3.py . Nobody else met the similiar issue?

jbhurat commented 5 years ago

Hi @colindog My apologies for the delayed response. I tried to reproduce the issue you mentioned with both Istanbul and raft, but I do not get stuck transactions. I am able to get an average TPS of around 270 with both istanbul and raft. Can you provide more details about your setup, how are you submitting the transactions and istanbul and raft configuration options.

I used chainhammer to perform my tests

jbhurat commented 5 years ago

Closing the ticket as there is no activity. Please open a new issue if you need any further assistance