Geth 1.10 txpool ordering

[patllc]

Rationale

With the latest update of geth 1.10, transactions with the same gas price are ordered by receive time, rather than by random order.

This receive time ordering may not be suitable to binance smart chain. Since BSC's validation mechanism is 21 validators taking turn deterministically to wrap transactions, deterministic transaction ordering can lead to below problems : 1) validators can GUARANTEED frontrun/ backrun pending pool transactions, as validators do not need to compete for POW. For example, a validator can guaranteed front running IDO starts at a pre-determined block number. 2) since latency becomes critical after the update, nodes have less incentives to include light nodes and nodes located remotely (like in New Zealand) as peers. Nodes are concentrated located and connected. This causes the nodes become more and more centralized and defeats the purpose of decentralization.

Also, I think this change is vital and wonder is that any reason not stating it in the change list?

Implementation

The origin of this change in ethereum mainnet is to avoid spam back running transactions occupying block spaces. To achieve the same goal in bsc, could we propose the below change instead? 1) sort the transaction order by receive time, with a noise added so that order is sorted by T + N(0,sigma) 2) revert the change if the spam issue not serious. Seems to me that the 60m gas limit is not fully utilized recently.

Welcome any thoughts!

[patllc]

Eth has flashbots and Mev-geth (I think bsc has not)

[myuniswap2000]

Eth has flashbots and Mev-geth (I think bsc has not)

hello, I don’t think anyone can cooperate with 21 nodes, but there are a small number of people who have a high probability of winning. They must have a way to get a position or static node. Is this inference correct?

[ghost]

how about this impact to frontrun-tx? t+2 packed

[Undead8]

The change in transaction ordering does not matter at all.

Validators can change the default ordering by modifying this repo, place transactions in whatever order they want, and hide that behaviour by giving their transactions a plausible gas price. If they are not doing this already, it is only a question of time before they do it. There are tens of millions of dollars to make by doing this (if not more) and there is no incentive to not do it. I certainly know that I would exploit transaction ordering to the maximum if I was a validator. Expecting that validators will renounce to that profit just to "act fairly" is wishful thinking.

Reputation is not a factor because: 1) You can plausibly deny manipulating the ordering by making sure that the transactions have a plausible gas price, and 2) Nobody cares anyways, except a few experts that probably all commented on this thread (regular BSC users do not care at all about transaction ordering and MEV).

On Ethereum, most of the searchers profit goes to miners through flashbot. On BSC, it most likely goes to validators too (or it will soon). It is just more opaque.

If your business relies on fair transaction ordering, maybe it's time to pivot...

[m-e-r-k-l-e-root]

Interesting thread

[Undead8]

@m-e-r-k-l-e-root I totally agree with you. My point was that the default ordering that is implemented in this repo does not matter because validators will simply change the default ordering (with a fork of this repo) to suite their MEV extraction needs.

Edit: It does not matter so much if preferential ordering is implemented by either (1) sorting by time received and giving static connections to some privileged nodes (what you accurately describe as the current BSC situation), or (2) validators manually reordering the transactions (probably not happening right now). The effect is that only a select few will be able to efficiently extract MEV and those select few will either pay the validators to do so or be the validators themselves. Eventually, most of the MEV profit will go to validators. As others pointed out, it is also a problem on Ethereum (falshbots) and Polygon, although implemented differently.

Very interesting thread by the way. I was wondering if others had noticed this and clearly I'm not alone.

[gillbates]

just impl a real time scanner, having found that: almost 99.9999% of txs related to swap/add/remove liquidity are front-runned by contracts with close relations to validators ... estimated 5-10M usd daily ... what a number there!

[misu200]

just impl a real time scanner, having found that: almost 99.9999% of txs related to swap/add/remove liquidity are front-runned by contracts with close relations to validators ... estimated 5-10M usd daily ... what a number there!

I am also frustrated with the ones that are better than me at snipping :) but it doesn't mean automatically "close relations to validators" . The edge in this game might not be about "relations to validators" .... but more about so many other stuff like: knowing how P2P works for geth... the locations and configurations for validators/sentry nodes...the setup for Binance JSON-RPC Endpoints ..customizing geth & gobal infrastucture...cloud latencies....Virtual Machine Networking Performance .Even finding validators locations doesn't guarantee success.... because the sentry nodes might be behind firewall also and have their p2p port (e.g. 30311) closed for incoming connections.

[ghost]

@patllc How do you even test if a node is validator/sentry node? You said that even if you know the location of each validator, you can't get close enough to defeat the profit makers, if you can get close to them, isn't that good enough?

@gillbates how were you able to deduce if an account/contact have relations to a validator?

@misu200 the problem is to minimize the latency to such validator/sentry nodes

[myuniswap2000]

just impl a real time scanner, having found that: almost 99.9999% of txs related to swap/add/remove liquidity are front-runned by contracts with close relations to validators ... estimated 5-10M usd daily ... what a number there!

I am also frustrated with the ones that are better than me at snipping :) but it doesn't mean automatically "close relations to validators" . The edge in this game might not be about "relations to validators" .... but more about so many other stuff like: knowing how P2P works for geth... the locations and configurations for validators/sentry nodes...the setup for Binance JSON-RPC Endpoints ..customizing geth & gobal infrastucture...cloud latencies....Virtual Machine Networking Performance .Even finding validators locations doesn't guarantee success.... because the sentry nodes might be behind firewall also and have their p2p port (e.g. 30311) closed for incoming connections.

I think you are right, can you explain the conditions you mentioned？ what means the setup for Binance JSON-RPC Endpoints?

[tino-web]

I'm wondering why not more people are talking about this. For a few months, t+1 is no longer possible and it's always t+2 or more, no matter the gas price. Did anything change except increasing txs and growing chain size? Are the devs working on a solution to get back to t+1?

[b10c77]

I'm wondering why not more people are talking about this. For a few months, t+1 is no longer possible and it's always t+2 or more, no matter the gas price. Did anything change except increasing txs and growing chain size? Are the devs working on a solution to get back to t+1?

AFAIK it's been pending block + 2 (mined after 2 blocks) for as long as I've used BSC.

[ktrout1111]

Agree with the OP.

Not only is this change unfair, it doesn't stop spamming either. Two of the "anointed ones" with unexplained powers to almost always beat everyone else to the punch here: https://bscscan.com/address/0xaf25d6494a341675186f975c4cbc081fdac05dbc and here: https://bscscan.com/address/0xfb921a3925e751103a50c50fa8805798cae65d75 are spamming 30 or more transactions each in a single block when they both spot a big haul.

@guagualvcha, @kyrie-yl, @j75689, @yutianwu - please - I and others implore you to roll back the transaction ordering algorithm back to the random order that it was, or at least some other method that is at least fair. Perhaps, as @patllc suggested, sorting the transactions of equal gas price by hash would reduce spam? It would incentivise arbitrageurs to "hold fire" on a single transaction until they have expended what they estimate to be enough hash power, rather than just immediately spamming dozens of transactions as soon as they spot a mark.

It would also provide a feedback mechanism: bots monitoring pending transactions would see the current "best" hash and know not to send their current work in progress if its hash is "worse". This would, in theory, drastically reduce bot spam. It would turn the competitive playing field into a traditional PoW hash battle - the burden of which is born by the bot operators instead of the network. Bot operators get a fair slice of the pie proportional to hash power invested, and regular users and node operators will no longer be bogged down with spam.

Although it has been correctly pointed out in this thread that validators are free to run whatever code they want, it does look like they just upgrade when a new version is released, with no conclusive evidence of direct MEV exploitation.

[scorring]

Interesting thread. I have developed a transactions scanner that catches all arbitrage attempts and observed that I most frequently could not catch the transaction hash that wins the arbitrage. Is my scanner buggy or are these hashes generated by validators ? I totally agree with the comments saying that validators would generate and reorder transactions to win the game, or doing so for friends. There are so much money there. If I had 10000 BNBs to stack, I would definitely do that. However, if this is true, why are we still seeing so much arbitrage attempts for one transaction? Sometimes more than 50 contracts calls after a swap? Is there a private battle between arbitrators friends? Do the validators want to occult theirs practices by spaming the pool?

[MaksimKel]

Hi guys! very interesting article, I regret that I did not see it earlier)

Is anyone else trying to create an arbitrage earning system at this time?

In my opinion, validators have nothing to do with it. I have been creating my script for arbitration for a long time and during this time I have had a lot of successful transactions. From receiving the window signal to committing the transaction, I spend 0.012ms, but even this is not enough for permanent success. I think it's all about strategy.

[taina0407]

Hi guys! very interesting article, I regret that I did not see it earlier)

Is anyone else trying to create an arbitrage earning system at this time?

In my opinion, validators have nothing to do with it. I have been creating my script for arbitration for a long time and during this time I have had a lot of successful transactions. From receiving the window signal to committing the transaction, I spend 0.012ms, but even this is not enough for permanent success. I think it's all about strategy.

@MaksimKel Can you share more information about your "system". I've build my own BOT extracting MEV, from signal new pending transaction to submit my owns will take around 10-50ms; but all of them are failed. Then I've checked bscscan the "arbitragable" transaction, and find out that the very next transaction index in the block will be another MEV bot transaction. Since mine are 50-100 transaction index behind. I'm thinking about of location where my node was hosted, I think that is because of server location but cound't find any information about validator location to verify my theory, so please at least share where did you host your system for reference if you don't mind.

Btw < 1ms signal delta was very impressive, congratulation!

[tomekozlowski]

@MaksimKel oh is that so? And how did you get past the t+2 problem?

[AlexKNM]

"transactions with the same gas price are ordered by receive time" - what is the issue if you can simply add more gas and be first in the queue, assuming, that validators do not rearrange their own transactions?

[AlexKNM]

Hi guys! very interesting article, I regret that I did not see it earlier) Is anyone else trying to create an arbitrage earning system at this time? In my opinion, validators have nothing to do with it. I have been creating my script for arbitration for a long time and during this time I have had a lot of successful transactions. From receiving the window signal to committing the transaction, I spend 0.012ms, but even this is not enough for permanent success. I think it's all about strategy.

@MaksimKel Can you share more information about your "system". I've build my own BOT extracting MEV, from signal new pending transaction to submit my owns will take around 10-50ms; but all of them are failed. Then I've checked bscscan the "arbitragable" transaction, and find out that the very next transaction index in the block will be another MEV bot transaction. Since mine are 50-100 transaction index behind. I'm thinking about of location where my node was hosted, I think that is because of server location but cound't find any information about validator location to verify my theory, so please at least share where did you host your system for reference if you don't mind.

Btw < 1ms signal delta was very impressive, congratulation!

where is your node is hosted?