Enable submission of many transactions to the same consensus node

[mok0230]

Problem

My understanding is that as of today, concurrent transactions from the same from address are not guaranteed to be sent to the same Consensus Node via JSON RPC Relay.

I am working on integrating the Hedera network to OpenZeppelin Defender. Users of Defender Relayers will often submit hundreds or even thousands of transactions to the network (with incrementing nonces) in a short period of time. In networks with mempools, this works fine - the transactions will all be ordered within the mempool.

This is my understanding of the current (likely) behavior in Hedera JSON RPC Relay (let's assume current nonce of this account is 5):

• Transaction 1 (with {nonce: 5}) is submitted to JSON RPC Relay, which routes to Consensus Node A
• Transaction 2 (with {nonce: 6}) is submitted to JSON RPC Relay, which routes to Consensus Node B, but B has not yet been gossiped to about Transaction 1 so it still has the account's nonce as 5. The transaction will not process and must be resubmitted at some later point once Consensus Node B (or whichever node receives the next transaction request) has been gossiped to about Transaction 1, which will prompt updating the account's nonce

This can end up significantly slowing down transaction speed due to the waiting period as described in the second bullet

Solution

Route all transactions from a given from address to the same Consensus Node

Alternatives

There could be other ways to route a transaction to a particular Consensus Node (there could be an additional field on the transaction for instance) but I don't think they would be as clean. And the big need here is related specifically to transactions coming from the same address because it's the nonce updating among Consensus Nodes that is a problem.

[spalladino]

Route all transactions from a given from address to the same Consensus Node

To be clear, we don't need to route all transactions to a specific consensus node, just consistently to the same one given the same sender address. So any method of stickiness (eg hashing the sender address and calculating its modulo on the number of consensus nodes available, as @mok0230 had suggested in internal design discussions) would work here.

[Daniel-K-Ivanov]

Thank you for the detailed description of the issue. The check for the account nonce is performed at consensus time and the ordering of the transactions is not based on the nonce but actually the median timestamp at which a majority of the consensus nodes have received/seen the specific transaction.

Roughly speaking, every ~300ms Consensus nodes are "finalising" on the sequence of a list of transactions. Once they asynchronously compute the sequence, each one of them passes the list of transactions to be executed (go through consensus). At this point, for every transaction Consensus Nodes are doing the nonce check (+ other prechecks) and then spin up the EVM to execute the opcodes.

That being said - It is questionable whether adding stickiness will mitigate the risk. We will come back with more information once we assess the problem.

[Nana-EC]

Thanks @mok0230 and @spalladino This isn't a feature request we're able to take on right now. I'll be moving this to the backlog for future re-triage as our roadmap develops.

However, you're welcome to open a PR at your convenience with a desired approach if you have one in mind. Ultimately, since there is no mem pool transactions are ordered based on consensus timestamp. As such the only way to ensure ordering at scale is by confirming consensus is reached on a transaction before submitting the transaction with +1 nonce.

Thanks