Random ordering of equally-priced transactions incentivises competitive spam

[livnev]

Current behaviour

In the algorithm that a miner uses to order pending transactions into a new block, go-ethereum chooses a random order for transactions from different senders which have the same gas price. I believe this randomisation happens here, due to the fact that iterating over a map in Go randomises the keys (n.b. I am not familiar at all with the go-ethereum code base so this is just my naive guess). If this is indeed due to the default Go behaviour, it is possible that this "feature" is an accident, and not introduced for any specific reason (I am keen to hear if this is the case).

The problem with it

It has long been widely known that transaction submitters will bid higher gas prices in order to be included earlier in a block, often bidding in competition with other unconfirmed transactions. This is known as "frontrunning" or PGA (priority gas auction). It is recently becoming more widely known that in addition to this familiar sort of competition, there is also a widely employed method called "backrunning", in which a transaction sender wishes to have their transaction ordered immediately after some unconfirmed "target transaction".

Common examples of "target transactions" which make this desirable are price oracle updates that allow liquidation transactions to be triggered immediately afterwards, or trades on AMM (automated market maker) exchanges such as Uniswap where one trade affects the price offered to subsequent trades. In order to maximise their changes of being mined immediately after their target, a typical backrunner will send many identical transactions, with gas price identical to that of the target transaction, sometimes from different accounts, in order to increase the chances that one of their transactions is ordered after the target but before any competitor.

In essence, instead of a PGA in which competitors for a profitable transaction end up paying a portion of MEV (miner extractable value) to miners in gas fees by using a high gas price, backrunners transfer MEV to miners by spending more gas, without increasing the gas price. Compared to PGAs, backrunning likely imposes a larger negative externality on the network by consuming the gas limit (i.e. using up network throughput with useless transactions), unlike PGAs which do not have a disproportionate effect on block gas utilisation. For example, a backrunner who is prepared to spend $1,000 to capture a profitable opportunity by sending 1,000 $1-gas transactions is chewing up much more of the network's resources than a frontrunner who spends $1,000 on one high priority transaction. Therefore, it would likely benefit the network if backrunning was converted into frontrunning when possible, as a "less harmful outlet" for MEV extraction (the general concept of MEV and its potential negative externalities is out of the scope of this issue).

Examples

There are countless examples occurring every day, on different smart contract systems. Just a few hours ago, this transaction which mined in block 10496844 was targeted, with many backrunning transactions targeting it appearing in the same block and in previous blocks (for the curious, this was the ultimate "winner"). There have been many even more egregious examples where the target transaction (and consequently any other unconfirmed transactions in the network at the same or lower gas price) was effectively delayed/censored for many blocks by the backrunners. It seems that today a non-negligible part of the block space is occupied by this activity.

Possible solutions

Backrunning seems largely to be made possible due to go-ethereum's method of randomly ordering identically-priced transactions. If transactions were deterministically ordered, or ordered by some other quantity like arrival time, backrunning in its current form would not be possible. If ordering by arrival time, low latency transaction propagation will be incentivised instead in similar scenarios (whether or not this has any harmful side-effects is unclear). If deterministically ordering (e.g. by transaction hash), then participants will be incentivised to "mine txhashes" (or similar), which would add a PoW-style game to transaction priority but at least would not use Ethereum network resources. There could also be other approaches to ordering transactions that do not incentivise spam.

It is also possible for smart contracts to be designed in a way that removes the incentives for backrunning. For example, a price oracle contract used for liquidation or trading can have price updates take effect only in the following block after a price update transaction. For illustration:

  function poke(uint256 wut) {
    was = val;
    val = wut;
    wen = block.number;
  }

  function read() {
    if (block.number > wen) return val;
    else return was;
  }

However, it is not always clear that a backrunning scenario might be incentivised, or how to prevent it, so it may be preferable for this to be addressed at the client level. Smart contract authors may wish to mitigate this on the smart contract level for the time being (backrunning typically deteriorates the QoS for users of a smart contract, since it can have the effect of delaying user transactions or delaying oracle updates, etc.).

Outstanding questions

• Was there a reason that random ordering for identically-priced transactions was chosen, or is this purely an implementation detail in go-ethereum?
• Are there disadvantages to the other possible tie-breaking methods for transaction ordering?

thanks to @palkeo, PhABCD, and others for their research on this topic

[holiman]

Thanks for the detailed write-up! > If deterministically ordering (e.g. by transaction hash), then participants will be incentivised to "mine txhashes" (or similar), which would add a PoW-style game to transaction priority but at least would not use Ethereum network resources. That sounds to me like a very nice idea, spontaneously. 

[Austin-Williams]

If Geth does move away from randomized choice, I would prefer FIFO instead of "lowest tx hash", because the latter benefits those who can afford a lot of parallel computational power.

FIFO benefits those with lower latency connections to mining nodes, but that arms-race has a limit (you can get a short fiber connection directly from your node to all mining nodes). But the arms-race for "lowest tx hash" has no limit -- you can always spend more money for a larger GPU cluster.

[PhABC]

From some preliminary analyses looking at contracts flagged as running these spam bots, it looks like the spamming of transactions from backrunning bots increased significantly since this April. In total, they reach a total of $2.5m in gas spent, worth about 38,000 blocks. Not all these transactions are spammed, but assuming a 5-10 spam transactions for every valid one, the number is still jarring. As of recently, almost 10% of all gas used in a day is used by these contracts.

You can see the data here ; https://explore.duneanalytics.com/public/dashboards/FFFpCKoE41bvFpESiyjUIBJfEMt4GoMFwcidNcAh

[hendrikhofstadt]

I think that hash ordering is not a solution as bots will still spam better hashes as they mine them. They cannot rely on what they see in their local mempool to determine whether they will win with their last submitted hash (especially since the new 2 stage propagation protocol).

Time based ordering sounds like a fair solution here even though as Lev pointed out there might be unknown side effects.

I also want to link the list of bots here to allow further research: https://gist.github.com/hendrikhofstadt/6165e6a1a9baf07453cb96edfc8e5ef6

These are all running arbitrage strategies. Liquidation opportunities can be much more lucrative than arbitrage opportunities. Bot operators will backrun the price oracle for opportunities of sometimes 10k+. There's a separate list of bots that are not included in my original analysis which specifically run backrunning on DyDx and also caused strong spikes at times.

[wjmelements]

Not all miners use vanilla geth but most do. So-long as enough miners continue to tie-break randomly there will be incentive to spam.

FIFO ordering prevents backrun arbitrage theft but encourges competitive nodes not to forward transactions from their competitors.

I would prefer for the tie to be broken by sender address; this would break the spam strategy which should be the goal here. Unlike transaction hash, you cannot mine a new sender in time because your transaction would not be accepted into the network without ETH in your account.

[livnev]

I think that hash ordering is not a solution as bots will still spam better hashes as they mine them. They cannot rely on what they see in their local mempool to determine whether they will win with their last submitted hash (especially since the new 2 stage propagation protocol).

@hendrikhofstadt Good point, it sounds like txhash ordering might still generate some spam, but maybe less spam.

I would prefer for the tie to be broken by sender address; this would break the spam strategy which should be the goal here. Unlike transaction hash, you cannot mine a new sender in time because your transaction would not be accepted into the network without ETH in your account.

@wjmelements If tie-breaking is by sender address, can't you still mine an address on demand and fund it with ETH using a higher gas price than the target, returning us to the same spam issue? The funding tx could land in an earlier block or higher in the same block as the target (remember that one can try to delay the execution of the target tx to a later block with spam). However in any case it seems like the optimal strategy might be to pre-mine a huge table of sender addresses so that you can always lookup a close one that is greater than the target, making this a PoW-style game which will be dominated by whoever has the biggest/most performant table.

[throwaway-anon]

Longtime "spam" bot operator here (don't love that label), just throwing in some thoughts:

Putting aside issues of fairness to competing bots, all of the proposed solutions either don't seem to solve the problem or introduce negative externalities. Ordering by sender address can be gamed using CREATE2 and spending compute to build a lookup table of favorable salts, as has been partially suggested. There is no advantage to generating and preloading regular dust addresses with ETH. (edit: Oops, of course regular dust addresses need to be generated. I think spam would be generated by moving funds around to increasingly more favorable addresses) There is a tradeoff though in how to spend this compute, and I'm not sure that is solves the underlying problem. I imagine that otherwise idle compute time would be spent building out a lookup table for predictable sender addresses (e.g. those that have a history of generating profitable opportunities, or that signal that they are likely to generate one). However, when a transaction that is profitable to backrun hits the txpool, I expect that compute will be reallocated to that address, and for bots to submit transactions with increasingly favorable sender addresses as they are discovered, getting back to the spam issue. Mining transaction hashes would be even worse, as they are more computationally intense to generate. It could be that it is trivial to predict which sender addresses generate opportunities, and that there is less advantage to trying to mine more favorable addresses while a profitable transaction sits in the mempool, in which case spam could be reduced. This could be interesting to model.

As @wjmelements pointed out, the other proposed solution of FIFO would probably result in competing nodes selectively routing transactions, but it would also incentivize a huge number of dummy nodes, and I'm not sure that such a thing is good for the health of the network. It's fairly trivial to cheaply deploy a dummy geth node that relays transactions, and that pretends to maintain the state of the blockchain, and if this can increase the chances of getting close to a miner, then you can bet it will be done at scale.

I recognize the issue here, but there is something intrinsically fair to the "lottery ticket" dynamics that currently exist, which doesn't lead to long term winner take all dynamics as in gas auctions. Maybe there are other solution that preserves these dynamics but free up block space?

And of course I am paid to say this, but I think this "spam" issue is not as harmful as has been suggested. What about the gas usage of foreign MLM/ponzi shcemes like MMM, Forsage, Lionshare, etc... that continue to clog the network? Surely it is much higher than that generated by these bots. Why should arbitrage and liquidation transactions be singled out, and not these other types of transactions, if we are all paying users? There is also the issue that @wjmelements raised that as long some miners don't upgrade their geth nodes, or use other node software that continues to randomly order, spam will persist. Regardless of what is decided, I don't think that a "solution" should be pushed through without consulting the wider community.

[MicahZoltu]

Thanks for the detailed write-up! > If deterministically ordering (e.g. by transaction hash), then participants will be incentivised to "mine txhashes" (or similar), which would add a PoW-style game to transaction priority but at least would not use Ethereum network resources. That sounds to me like a very nice idea, spontaneously.

This still results in spam as all of the competitors would generate one transaction, submit it to the network, then hash grind until they found a better transaction than the current "best". Once they did, they would submit that and go back to hash grinding. Even if you are currently winning, you still are incentivized to continue to hash grind and beat your own transaction just in case you have a competitor who has a better transaction on the wire that you haven't seen yet. Looks like others already brought this up.

[MicahZoltu]

Time based ordering sounds like a fair solution here even though as Lev pointed out there might be unknown side effects.

We should not be afraid of the dark. We know for certain the current solution incentivizes bad behavior, unless someone can provide an attack vector for the new solution then we should prefer an unknown here to a known bad.

FIFO ordering prevents backrun arbitrage theft but encourges competitive nodes not to forward transactions from their competitors.

This is already true across the board. When you are running a bot for anything you want to censor your competitors. Ethereum is as censorship resistant as we can make it but not perfect. Switching from random to FIFO won't change the censorship resistance properties of Ethereum nor meaningfully change incentives for bots.

What people are discussing here is basically that "eclipse attacks are possible against Ethereum" which I believe is pretty well known. Switching to FIFO doesn't create the opportunity to eclipse attack and switching to FIFO doesn't incentivize eclipse attacks any more than they already are. What we are comparing here isn't FIFO vs some idealistic solution. We are comparing FIFO to the current solution with known problems. This is definitely not a time where we should let perfect get in the way of good.

What about the gas usage of foreign MLM/ponzi shcemes like MMM, Forsage, Lionshare, etc... that continue to clog the network? Surely it is much higher than that generated by these bots. Why should arbitrage and liquidation transactions be singled out, and not these other types of transactions, if we are all paying users?

We don't have an easy and censorship resistant way to reduce the spam generated by the former, but we do have an easy and censorship resistant way to reduce the spam of the latter. The goal of Ethereum isn't to get as many fee generating "paying users" as possible. The goal of Ethereum is provide a useful service to a broad demographic of people. Any opportunity we get to improve the quality of service for those users we should take.

[Agusx1211]

I would prefer for the tie to be broken by sender address; this would break the spam strategy which should be the goal here. Unlike transaction hash, you cannot mine a new sender in time because your transaction would not be accepted into the network without ETH in your account.

This doesn't removes the ability to brute-force the order of a front run transaction by spamming the network with transactions from random senders, but it adds another vector for doing that but with some information of the resulting order of such transactions.

A front runner operator can keep front-running but without spamming the network as much, because now they know what's their best possible candidate if they want a given slot on the block.

If the order is totally deterministic (Ej: low addresses go first, high address go last) then front-runners are likely going to have prefunded addresses that are distributed through the block, they could keep reusing those address so there is no need to spam the network funding and defunding EOAs. They could find better EOAs, but once they have fund relatively good addresses it's highly unlikely that they are going to find a better candidate in the span of a single TX.

I think that should shift the scenario from multiple txs from a single bot, competing with itself to the best tx from each bot, competing with each other.

[MicahZoltu]

I agree with your assessment @Agusx1211 that it will result in 1 transaction per bot.

I believe FIFO will have a similar behavior, but it also incentivizes nodes to be well connected (a good thing for the network) rather than hash grinding (and potentially seeding) a large number of accounts (not good for the network).

[Agusx1211]

sender address could also reduce the competition between bots, because the front-runner knows their best candidate and they could also know the best candidate of the other bots, if they know that there is a better candidate already on the mempool there is no point on sending a tx that's bound to go second.

FIFO seems fine to me, but I worry that it may not incentivise well connected nodes, but it instead incentivises nodes that try to eclipse miners (a few milliseconds should be enough). The front-runner could have a lot of nodes that behave "correctly" and when they want to send a tx they could tell those nodes to stop relaying competing txs.

Switching to FIFO doesn't create the opportunity to eclipse attack and switching to FIFO doesn't incentivize eclipse attacks any more than they already are.

@MicahZoltu why do you think that's the case?

[hendrikhofstadt]

I think "network neutrality" wise FIFO is more fair but tend to agree that the risk of more strongly incentivizing eclipse attacks is not negligible. However I'm pretty sure this would not be a major issue because Miners (could) sell direct interconnects to their nodes which would quickly reduce the ROI for anyone trying to eclipse / strategically DoS nodes.

I agree with your thoughts on the effective outcome of sender sorting. I think @livnev summed it up very well saying that the endgame would be having a huge table of premined addresses ready.

However the way the tx pool is currently implemented these txs would not be accepted until the sender account is funded (which would need to happen 1 block in advance). This implementation detail makes sense to prevent DoS on the mempool. https://github.com/ethereum/go-ethereum/blob/6c9f040ebeafcc680b0c457e6f4886e2bca32527/core/tx_pool.go#L546-L548

This means that at the current state of geth, bot operators would need to prefund a sufficiently large amount of well distributed addresses with sufficient ETH because they can't just fund them at a higher gas price in the same block as the victim (due to the check above). That however creates an incentive to spam high gas price txs to consecutively push the victim tx to a later block and get enough time to fund a better account (potentially from a premined pool).

I agree that this could significantly reduce the amount of spam for smaller and medium sized opportunities. However for larger opportunities (there have been ones worth 100ETH+) it might create an incentive to fill whole blocks to buy more time for account funding. Still that would just be a short spike and should not affect the gas price for more than a couple of blocks.

[MicahZoltu]

@Agusx1211 You make a good point that with FIFO there is incentive to do a partial eclipse attack where you just need to make the route from your competitor to the node slower, but you don't actually need to fully censor them. Even with that, I feel like that is still a win for Ethereum since an effective eclipse node needs to appear useful to peers, which (if things are behaving appropriately otherwise) means that it is propagating blocks, transactions, etc. generally speaking.

So while I think you are right that FIFO does incentivize partial eclipse attacks, I'm not convinced this is unhealthy for the network (and arguably could be healthy for it).

[MicahZoltu]

Another thing to keep in mind with sender address sorting is that if the working strategy for a particular bot is to pre-fund accounts, then you end up with the problem of state bloat as participants are incentivized to keep gas money in a wide variety of accounts.

[Agusx1211]

That however creates an incentive to spam high gas price txs to consecutively push the victim tx to a later block and get enough time to fund a better account (potentially from a premined pool).

Isn't this the case with FIFO too? An attacker could try to push the victim to a later block until they are confident about they being on the right spot on the mempool (maybe the front-runner doesn't know it, because it doesn't knows the order on which the miner received the txs).

Also, opportunities worth 100ETH+ are a huge beast on their own, those could cause re-orgs by making the miners compete with each other to be the one to claim it.

[MicahZoltu]

An attacker could try to push the victim to a later block until they are confident about they being on the right spot on the mempool (maybe the front-runner doesn't know it, because it doesn't knows the order on which the miner received the txs).

FIFO would be absolute time that a node saw a transaction for the first time. This means that once you have lost, you have lost forever unless the miner drops a transaction from the transaction pool (either the current winner or the transaction everyone is backrunning).

[Agusx1211]

Yep, I think you are right. Unless the front-runner finds a way of delaying the transaction for miner A while allowing the transaction on miner B (assuming that miner B has the mempool sorted in such a way that benefits the bot). But I don't think that's possible.

Still, I think that if it comes to that, incentives for the miners start to appear. And the competition shifts from who can manipulate the mempool to who can mine those txs on the longest chain.

[hendrikhofstadt]

@Agusx1211 You're making a very good point with pushing the tx between miners. You might also "collude" with one miner and try to delay the tx (by filling blocks) until that miner can take it (i.e. mines a candidate) but I guess we already have that dynamic right now. So I would consider that out of scope for this particular issue.

[brockelmore]

What spammers really care about is relative positioning. They don't care about the entire block order, generally just relative ordering y directly after x.

One idea that was throw around in discussion, was highest gas price pays to signal relative ordering. i.e. top gas price calldata is a list of [x -> y] tx hashes, with y.gasPrice <= x.gasPrice and some other constraints. In general, you can think of the gas prices in the mempool like this : [[band 1], [band 2], [band 3]]. If you have the highest gas price, you have the right to order txs inside a band explicitly (and by extension, order top of band 2 and bottom of band 3).

This lets you preserve most of the ordering work whenever a new top gas price comes in. i.e. when a new top gas price saying order band 2 to be x -> y, you only have to update band 2. As for miner participation, they would have no obligation in this, so participants bear all risk, but if most miners run stock geth, then it would work to reduce spamming by essentially turning everything into a PGA.

[hendrikhofstadt]

@brockelmore Interesting! This sounds like a "segmented" version of MEV Auctions.

What we need to consider is that this would need to be a more complex coordinated effort between all client developers and due to the impact of this change probably also the core devs. I feel like this would be a candidate for an EIP vs a quick fix.

[MicahZoltu]

If we want to support this sort of thing we could add a dependsOn field to transactions that, if present, would indicate the transaction is only valid after a particular transaction. This would turn back running into price auction just like front-running. If we wanted we could further add a field (or perhaps just a bit somewhere) for atomicWith meaning the transaction MUST be run immediately after the target transaction or it is not valid.

The nice thing about atomicWith is that it means all of the loser transactions would not consume any gas. Winner pays gas price bid, losers pay nothing (get dropped from pool as invalid transactions). I think this would be the best solution in terms of minimizing the gas spent on these competitions, and it would be nice for some other things as well (though, many of the "other" things are solved with EIP-2711).

[throwaway-anon]

The nice thing about atomicWith is that it means all of the loser transactions would not consume any gas. Winner pays gas price bid, losers pay nothing (get dropped from pool as invalid transactions). I think this would be the best solution in terms of minimizing the gas spent on these competitions, and it would be nice for some other things as well (though, many of the "other" things are solved with EIP-2711).

This will end up converting all value that can be extracted through arbitrage/liquidation/front-running to miner profits once competing bots can no longer further optimize for gas usage. I would fully support this if the value went towards making Ethereum better, or back to Ethereum stakeholders (i.e. block producers in eth2), but PoW miners clearly don't have the best interests of the network in mind as evidenced by the recent gas limit increase that was done without consulting community members.

Unless I'm missing something, I just realized that FIFO still results in spam when backrunning. Just because you have observed a transaction in your mempool doesn't mean that a miner has it in their pool, and so you still need to make sure that your transaction arrives after the transaction of interest, and not before. Perhaps the ordering in the dependsOn solution suggested by @MicahZoltu could be determined by FIFO, and not gas price? This seems to solve the problem while still compensating arbitrageurs.

[hendrikhofstadt]

@throwaway-anon Good point on the FIFO spam. Still that would be significantly less spam as it would just be distributed over a couple of ms but it could still be quite a bit if there are many bots.

Considering the above points the sender sorting appears to be the most solid "quick fix" so far imo.

@MicahZoltu 's dependsOn concept would indeed ultimately transfer most value to the miner. But to be fair that's also the case with any kind of PGA or spam. Extrapolating the numbers that @PhABC presented above, the increasing amount of spam will ultimately lead to most rewards being lost to miners while causing increased load on the chain.

So the endgame is theoretically the same, just that with the proposed change the load on the chain is reduced significantly. Also bot operators would be incentivized to actually take positions (to reduce gas usage) and potentially source off-chain (or other chain) liquidity allowing for more sophisticated strategies vs super low-risk atomic arbitrage.

[throwaway-anon]

@throwaway-anon Good point on the FIFO spam. Still that would be significantly less spam as it would just be distributed over a couple of ms but it could still be quite a bit if there are many bots.

Considering the above points the sender sorting appears to be the most solid "quick fix" so far imo.

I'm still not convinced by this. I think you're underestimating the level of constant address churn spam that this would create in trying to predict which sender addresses will generate backrunnable transactions. You also have the issue that you can still spam to delay a transaction from being mined in order to get a winning address funded, as I think others pointed out above.

@MicahZoltu 's dependsOn concept would indeed ultimately transfer most value to the miner. But to be fair that's also the case with any kind of PGA or spam. Extrapolating the numbers that @PhABC presented above, the increasing amount of spam will ultimately lead to most rewards being lost to miners while causing increased load on the chain.

Right, I agree that in the long run that the current system of random ordering does not yield positive expected value for the arbitrageurs, and just leads to spam. But the question is whether miners or arbitrageurs should be compensated with the new rules. What is the argument for tie-breaking/ordering dependsOn by gas price versus FIFO?

[AFDudley]

I suggest adding dependsOn and allowing that to bind to contract state as well, as a later addition. Binding to contract state is more complicated, but could contribute to preventing ordering manipulation as well.

[hendrikhofstadt]

@AFDudley I'm concerned that binding to contract state might make this prone to DoS. With the new snapshot backend complexity of accessing state has decreased but it's effectively still a free CALL (because it loads the other contract's state) + SLOAD (because it reads an element from state) at 0 cost.

I'm still not convinced by this. I think you're underestimating the level of constant address churn spam that this would create in trying to predict which sender addresses will generate backrunnable transactions. You also have the issue that you can still spam to delay a transaction from being mined in order to get a winning address funded, as I think others pointed out above.

I agree with you. But I think there's a ceiling of how many addresses someone would prefund to keep profitable. Even if it's 50k addresses, the impact on ETH state would still be considerably small. I assume that the most capital effective strategy would be if bot operators prefunded addresses close to "big fish" and left all other opportunities to PGAs in the next block. But I might underestimate that plus the aftermath of thousands of dust accounts being left.

On the point of "delaying transactions" I'd still go with what I said before because this is already an issue if you consider that a arbitrageur might collude with a miner. It would of course increase the incentive to run such an attack on larger opportunities but it would come at a high cost because while you delay the tx someone else might have mined a better address. So my assumption was that the risk would exceed the potential profit in most cases and the cases that are left would be negligible exceptions with little impact on the fee market.

After all the sender sorting would just be a "band aid" / temporary solution until concepts like dependsOn are implemented.

[livnev]

I feel like changes to the protocol (like adding a field to transactions) are definitely out of scope here: if you want to propose something you can make an EIP for that. I think the scope of this issue should be restricted to "easy" fixes specific to go-ethereum to rectify what seems to be an implementation detail in this specific client that has accidentally incentivised spam.

[PhABC]

FIFO spam is indeed concerning, but I think there are more cost effective networking strategies for backrunners. For instance, you could broadcast the victim's tx right before yours to all your peers, reliably putting your transaction right behind it. Doing this across the network somewhat guarantees your tx will be after the victim's, assuming your tx isn't re-broadcasted alone in a faster and more significant fashion.

Also, considering how small of a change this is, I don't see the harm in trying FIFO and seeing the impact it has on the backrunning market over the coming weeks/months, until something better comes along. We can't do much worse than a random sort anyway.

[Agusx1211]

I don't see the harm in trying FIFO and seeing the impact it has on the backrunning market over the coming weeks/months

Is there any data on how often miners update their nodes? I guess not often because they are probably prioritising stability over performance.

[MicahZoltu]

If we get the change in now, they'll get it by next hardfork at the latest.

[AFDudley]

I completely agree with @livnev the question right now is regarding simple changes to geth.

As I understand it the two options are "sort price buckets by first seen" and "sort price buckets by sender address"

The con of "FIFO", is that provides greater incentives for existing games between miners and bribery of miners. The con of "Sender address" is that it introduces a new set of games that are not well understood.

If I've summarized that correctly, I think FIFO makes more sense because it, or something very similar, was already implemented in Parity and we have a better idea of what to expect. I think both are short term fixes and in the long term these sorts of problems are solved with new transaction types, which again are completely of scope for this issue.

I suggest we submit PRs for both sorting strategies and apply one of them as the default in the soonest geth release possible. We should also consider making the sorting strategy configurable within geth, helping miners pick sane configurations instead of leaving it up to them to implement their own custom solutions which may inadvertently undermine overall network health.

[cheeselord1]

From some preliminary analyses looking at contracts flagged as running these spam bots, it looks like the spamming of transactions from backrunning bots increased significantly since this April. In total, they reach a total of $2.5m in gas spent, worth about 38,000 blocks. Not all these transactions are spammed, but assuming a 5-10 spam transactions for every valid one, the number is still jarring. As of recently, almost 10% of all gas used in a day is used by these contracts.

You can see the data here ; https://explore.duneanalytics.com/public/dashboards/FFFpCKoE41bvFpESiyjUIBJfEMt4GoMFwcidNcAh

Gas used by backrunning bots is increasing exponentially and the upper limit is only set by the amount of profit opportunity. As Ethereum DEX volumes increase, backrunning will continue consuming block space until most everyday users are priced out of the network.

Getting this added to Geth and implemented by mining pools ASAP would be huge. Even if a minority of hashpower switched to FIFO ordering, that would be enough to cap backrunning growth until the next hardfork forced everyone to update geth

[lightuponlight]

This is what the spam looks like in an application. All these orders appeared and then vanished the next block. 10 transactions worth of gas burned here. I see this on Oasis a lot these days as I MM there:

[adamschmideg]

Fixed by #21358

[onicrom]

I realize this issue is closed but I feel like this thread is a good place to post a related idea. Let me know if it is more appropriate to open a new issue.

What if we ordered the transactions with the same gas by senders, randomly selecting a character in [a-fA-F0-9] and then order sequentially based on address.

e.g. "f" is randomly selected. all sender 0xf addresses ordered first, then 0xF, then 0x0....0x9, then 0xa. If there are duplicates within (more than one 0xf) process them sequentially thereafter (0xfa, then 0xfb ...)

[MicahZoltu]

Any non-deterministic (in advance) sorting incentivizes spamming the mempool with many transactions to increase the odds of your transaction being in front of others.

The best we can do is incentivize the least bad, or ideally best, behavior. In this case, that is to get your transaction distributed across the network first, since that encourages good network connectivity as a side effect at least.

[onicrom]

That makes sense, but doesn't the randomness factor make it very difficult for someone to ensure a specific transaction is processed before or after others.

I wonder what attack is less bad for the network and/or less expensive for the attacker. Spinning up a number of nodes to increase their odds of being FI(FO) vs spamming the network with multiple sets of 22 transactions.

[MicahZoltu]

Adding nodes to the network is generally considered a good thing, while spamming transactions is definitely a bad thing for multiple reasons (primarily, it wastes a lot of gas and thus block space). Even if adding nodes is less expensive, it is generally a boon for the ecosystem and thus a positive side effect.

[onicrom]

Gotcha. When is adding nodes a bad thing, or what could someone do to a node to make having multiple instances running a bad thing?

Also, thanks very much for your responses.

[MicahZoltu]

Adding nodes is generally not a bad thing, especially if they are running stock software. An exception would be someone attempting an eclipse attack, or someone running maliciously modified software that tries to exploit some aspect of the network's design.