Open CodeForcer opened 3 years ago
Can only second that, there is a lot of really interesting innovation going on around gasless tech (e.g. Alchemist, Visor) which are seriously hampered by these limitations in MetaMask.
As we all know, high gas costs are one of the biggest barriers to entry for onboarding the next million users onto Ethereum. If MetaMask were to work with the teams building gasless tech, it could have a large positive impact on the entire ecosystem to allow more people to transact either entirely without gas, or at least have more flexibility for who's going to pay it.
P.S. I'd also add https://github.com/MetaMask/metamask-extension/issues/10240 to this list
We'll probably want to break this up into two issues because there are two proposals (really kindof three):
Since we already have issues for two of those, it might be useful to use this thread to discuss strategies for mitigating MEV. This might also be a good use for our new community forum, which supports threading and comment voting, since this could definitely be a big topic: https://community.metamask.io/
Using signTransaction is obviously a popular approach right now, but for the reasons on #7711, I think we should at least consider other approaches before accepting those impacts.
A couple proposals:
signTransaction
type method that submits the tx with a non-blocking multi-nonce.I'm also eager to hear of other ways that your services could be provided without MetaMask needing to trust every single website to manage the user's nonces for them.
Generally, I am in favor of minimizing the number of "jobs" wallets take on. In this case, metamask is trying to do key management + transaction execution lifecycle. I am of the opinion metamask should focus on key management and allow applications handle transaction execution lifecycle.
Ethereum is moving in a direction where there will be multiple alternative to the p2p txpool for delivering transactions to miners. Each channel suits different use cases which the wallet is unlikely to be aware of. Already several solutions like Flashbots, Taichi, Chainlink FSS, and Pokt Network provide alternative delivery mechanisms but are nearly impossible to use with metamask. This is why a long term solution needs to decouple key management from transaction execution.
This is a critical issue today as metamask users are currently loosing millions of dollars every day because their transactions are not protected from front-running.
IMO the easiest way to ship support for these alternative delivery mechanisms would be to enable eth_signTransaction
RPC. This could be made and "advanced" feature which a user explicitly enables. Metamask would cache a list of all transaction hashes produced through eth_signTransaction
and eth_sendTransaction
and warn the user if they are attempting to overwrite an existing nonce.
- Provide a method for adding a custom transaction relay to the wallet (avoids the problem of requiring every site to correctly implement nonce management)
While this alternative solution seems to fix the issue and is on the surface similar to adding support for alternative networks with a chainID of 1 to wallet_addEthereumChain
. In my opinion, it is a terrible option as any new rpc format supported by these new networks (like atomic transaction bundles) would need to lobby metamask to support their new features.
I'm also eager to hear of other ways that your services could be provided without MetaMask needing to trust every single website to manage the user's nonces for them.
While I understand the reasoning at the time was to stop nonce mismanagement, enabling eth_signTransaction
is a very complete solution which would enable MetaMask to support every conceivable gasless and front-running protected application (at least all the ones which currently exist). As @thegostep said there's no reason this has to be a standard feature, the user could be required to consent to it being enabled on each website which requests it.
As an intermediate solution, enabling 0 GWEI transactions would at least give us a workaround, as we can connect the user to a custom RPC from the backend to capture and bundle the 0 GWEI transactions. This isn't ideal or as good as the first fix, but it's an unblocker which at least provides some way for MetaMask to be supported.
Sorry for the delay here, we're juggling quite a few things at once.
As @thegostep said there's no reason this has to be a standard feature, the user could be required to consent to it being enabled on each website which requests it.
I think I agree with this. Our EIP-2255 permissions system makes it very easy for us to add a sensitive method behind a login permission for the user, so signTransaction
could fit well for this.
My earlier disagreement was that "letting any site" perform this action was dangerous, but "letting sites that are willing to disclaim the risk to their users" can definitely be made much more safely.
A couple drafts for the permission might be:
Happy to get other permissions wordings (or UX flows for when they get stuck) out there.
Any update on this please?
- This site will be able to relay your transactions instead of MetaMask. This can result in transactions being stuck, and using the "Speed up" or "Cancel" functions will override these transactions.
Some protocols offer free cancellation of unmined transactions via their protocol, for example BackRunMe ( bloXroute ). I'm not sure if Flashbots does this as well. It might be beyond the scope of MM to support this, but as a user I can tell you it would be a great benefit.
Updates on this?
I think this is pretty viable, we're looking at it as a plausible Q3 priority right now.
- This site will be able to relay your transactions instead of MetaMask. This can result in transactions being stuck, and using the "Speed up" or "Cancel" functions will override these transactions.
Some protocols offer free cancellation of unmined transactions via their protocol, for example BackRunMe ( bloXroute ). I'm not sure if Flashbots does this as well. It might be beyond the scope of MM to support this, but as a user I can tell you it would be a great benefit.
Yes flashbots does also allow this
Currently, MetaMask has 2 huge flaws which block the development of gasless tech and front-running protection for users:
1. The inability to sign transactions without also broadcasting them (first raised in #3475). Relays which protect users from front-running require bundles of signed transaction to be submitted to them (ie flashbots which has 60% of network hashpower right now). Unlike other wallet providers, MetaMask refuses to just sign a transaction without also attempting broadcast. I understand the reasoning for this at the time, but the situation on Ethereum network is now very different from before, and this is a common user requirement. A huge % of blocks now contain bundle-style transactions, and there are a bunch of white-hat tools being built around this need (https://flashbots.tools/)
2. Exception thrown by MetaMask when submitting 0 GWEI transactions A potential workaround for 1 is to connect MetaMask to a custom backend RPC to capture the serialised transaction and increment nonces for multiple user tx's in a bundle. However, MetaMask actually throws exceptions on any TX prices under 1 GWEI. This issue makes it impossible for gasless applications to support MetaMask.
Unfortunately, as part of a team working on next-generation gasless technology, we are in the difficult situation of having to recommend our users away from MetaMask, and instead onto other wallet providers which don't have this limitation. This is a tough move to make considering the popularity of MetaMask and our personal support for the project, but unfortunately in its current form MetaMask architecture blocks support for front-running protection and gasless transactions.
As the situation on Ethereum is different to when decisions such as #3475 were taken, can we revisit this issue and see if there is appetite in the MetaMask team for change. There are a lot of projects and teams now building in the gasless/privacy sphere, and without the issues outlined above being addressed there is a risk of MetaMask no longer being the de-facto wallet for next-generation applications.
side note: it is possible to sign transactions without broadcasting. use eth_sign on the keccak256 of the rlp transaction, and use the v r s from that to encode the full rlp tx
Currently, MetaMask has 2 huge flaws which block the development of gasless tech and front-running protection for users: 1. The inability to sign transactions without also broadcasting them (first raised in #3475). Relays which protect users from front-running require bundles of signed transaction to be submitted to them (ie flashbots which has 60% of network hashpower right now). Unlike other wallet providers, MetaMask refuses to just sign a transaction without also attempting broadcast. I understand the reasoning for this at the time, but the situation on Ethereum network is now very different from before, and this is a common user requirement. A huge % of blocks now contain bundle-style transactions, and there are a bunch of white-hat tools being built around this need (https://flashbots.tools/) 2. Exception thrown by MetaMask when submitting 0 GWEI transactions A potential workaround for 1 is to connect MetaMask to a custom backend RPC to capture the serialised transaction and increment nonces for multiple user tx's in a bundle. However, MetaMask actually throws exceptions on any TX prices under 1 GWEI. This issue makes it impossible for gasless applications to support MetaMask. Unfortunately, as part of a team working on next-generation gasless technology, we are in the difficult situation of having to recommend our users away from MetaMask, and instead onto other wallet providers which don't have this limitation. This is a tough move to make considering the popularity of MetaMask and our personal support for the project, but unfortunately in its current form MetaMask architecture blocks support for front-running protection and gasless transactions. As the situation on Ethereum is different to when decisions such as #3475 were taken, can we revisit this issue and see if there is appetite in the MetaMask team for change. There are a lot of projects and teams now building in the gasless/privacy sphere, and without the issues outlined above being addressed there is a risk of MetaMask no longer being the de-facto wallet for next-generation applications.
side note: it is possible to sign transactions without broadcasting. use eth_sign on the keccak256 of the rlp transaction, and use the v r s from that to encode the full rlp tx
I can speak for most of the people who replied here to say that we are actually very well aware of this, and it's how we are currently doing things. But it's a workaround and it's not a good look and neither is it a good approach.
Is it done yet?! Or is Metamask getting front-runned by other wallets ..
You may as well throw in counter arguments as why you are not implementing this for the sake of people's money.
please enable that!
+1, please help fix that.
+1. I wish we could get this done as soon as possible.
Still an open issue?!
Please fix it ASAP. It is the only reason it prevents me using the DEX. Why the delay???
Can you give us a firmer timeline? Q3 could be late September which is really late.
Is there a work around this issue? Metamask is not going out of their way to help us any time soon. If MIST wants adoption, we need to the trezor hardware wallet to work as soon as possible.
Problem that I face and use-case(https://github.com/mevalphaleak/BetaRPC-setup/blob/main/README.md): When someone submits a transaction(say NFT purchase) if this is a failing transaction and no bundle was included on-chain(Too late / not enough gas price / anything else) What's the best way to communicate back to MetaMask that transaction failed/discarded but it(MetaMask) shouldn't increase account nonce? Unless transaction is included on-chain MetaMask always shows it in "pending" state.
What I tried: 1) Users can cancel their transaction at any gas price(cancellation is not MEV/failure sensitive so it just goes to public mem-pool and "unstucks" MetaMask). Downside - it costs some ETH to cancel a tx without any purpose really. 2) People can use custom nonce. Downside - 99% of apes won't do it because they won't be able to find the option deep inside metamask 3) People can use "Reset Account" button in MetaMask which drops cached pending transaction. Downside - terrible UX same as (2) 4) My endpoint can respond with "failed" status for eth_getTransactionReceipt request of MetaMask then transaction is correctly marked as failed and removed from cache, but sadly MetaMask increases account nonce in this situation even if eth_getTransactionCount stays exactly the same for account.
Is there something that server can respond back to MetaMask which marks transaction as "failed/discarded" but keeps "nonce" exactly the same?
Problem that I face and use-case(https://github.com/mevalphaleak/BetaRPC-setup/blob/main/README.md): When someone submits a transaction(say NFT purchase) if this is a failing transaction and no bundle was included on-chain(Too late / not enough gas price / anything else) What's the best way to communicate back to MetaMask that transaction failed/discarded but it(MetaMask) shouldn't increase account nonce? Unless transaction is included on-chain MetaMask always shows it in "pending" state.
What I tried:
- Users can cancel their transaction at any gas price(cancellation is not MEV/failure sensitive so it just goes to public mem-pool and "unstucks" MetaMask). Downside - it costs some ETH to cancel a tx without any purpose really.
- People can use custom nonce. Downside - 99% of apes won't do it because they won't be able to find the option deep inside metamask
- People can use "Reset Account" button in MetaMask which drops cached pending transaction. Downside - terrible UX same as (2)
- My endpoint can respond with "failed" status for eth_getTransactionReceipt request of MetaMask then transaction is correctly marked as failed and removed from cache, but sadly MetaMask increases account nonce in this situation even if eth_getTransactionCount stays exactly the same for account.
Is there something that server can respond back to MetaMask which marks transaction as "failed/discarded" but keeps "nonce" exactly the same?
for the bundle monitoring have a RESTful API that users can query with their account address to check status, it can be integrated into something like blocknative's notify.js library, this is helpful especially if you are connecting to multiple different RPC endpoints that are not public.
for the bundle monitoring have a RESTful API that users can query with their account address to check status, it can be integrated into something like blocknative's notify.js library, this is helpful especially if you are connecting to multiple different RPC endpoints that are not public.
But how will this help with "discarded" transactions and use case I've explained above?
Yes it's easy to provide status of bundle and respond that it's "discarded" the problem that there's no such status in MetaMask.
Responding with "failed"(via eth_getTransactionReceipt
) always increases account nonce
Responding with "success" keeps transaction in "pending" state for metamask and users have to know about "account nonce" to manually set it for next transaction.
I want ability to signal to MetaMask that transaction was "Dropped" without increasing account nonce.
Hey Guys
Professional MetaMask specialist required
I’m looking to front run my MM wallet to avoid tokens being drained by a scammer that somehow hacked into my IMac and now has private keys (not sees phrase) Currently wallet has minimal ETH, have staked tokens that are untouched (on an ETH network. BSC network is also compromised and also expecting a large ( for me) airdrop in a new network (which I have added to the wallet.
Question is: can someone point me in the right direction to assist - happy to pay good fees. Harry Denley of MyCrypto and Scott Bigelow, Flashbots were suggested by MetaMask however neither have responded and it’s almost crunch time
Pete Hood Australia
@hoodmelb Id probably make a new issue for this. This seems totally unrelated.
Also my word of advise is to not build this, and instead practice good cybersecurity hygiene.
@hoodmelb Id probably make a new issue for this. This seems totally unrelated.
Also my word of advise is to not build this, and instead practice good cybersecurity hygiene.
Yeah thanks... the tokens have already left the building :(
Starting with a new computer
for the bundle monitoring have a RESTful API that users can query with their account address to check status, it can be integrated into something like blocknative's notify.js library, this is helpful especially if you are connecting to multiple different RPC endpoints that are not public.
But how will this help with "discarded" transactions and use case I've explained above? Yes it's easy to provide status of bundle and respond that it's "discarded" the problem that there's no such status in MetaMask. Responding with "failed"(via
eth_getTransactionReceipt
) always increases account nonce Responding with "success" keeps transaction in "pending" state for metamask and users have to know about "account nonce" to manually set it for next transaction.I want ability to signal to MetaMask that transaction was "Dropped" without increasing account nonce.
@mevalphaleak were you able to find a solution for signaling a "dropped" transaction without increasing the nonce?
I suppose the RPC could return a nonce+1, when metamask marks the transaction as "dropped" the RPC can resume returning the correct nonce.
Curious what your solution is.
@danfinlay
Flashbots & mistX have worked together to build an RPC that relays transactions directly to miners via Flashbots, avoiding the mempool entirely. This presents a few challenges around the UX for transaction states in MetaMask.
Transactions that fail via the Flashbots relay never appear on-chain thus keeping a "pending" state in MetaMask and triggering continuous retries. The RPC should be able to signal to MetaMask that a transaction has been dropped in a standardized way. Could be a specific error that the RPC responds from calls to eth_getTransactionReceipt
or eth_sendRawTransaction
.
Additionally, transactions that are relayed to Flashbots can be cancelled without sending a new transaction that lands on-chain with the same nonce. If MetaMask allows the RPC to signal a transaction is "dropped", both the cancellation and transaction could be "dropped" via the RPC giving the user a UX of free cancellations as well as failures.
Can we work together to come up with a standardized RPC response to signal that a transactions as been dropped?
Also, has there been any progress on allowing eth_signTransaction
?
for the bundle monitoring have a RESTful API that users can query with their account address to check status, it can be integrated into something like blocknative's notify.js library, this is helpful especially if you are connecting to multiple different RPC endpoints that are not public.
But how will this help with "discarded" transactions and use case I've explained above? Yes it's easy to provide status of bundle and respond that it's "discarded" the problem that there's no such status in MetaMask. Responding with "failed"(via
eth_getTransactionReceipt
) always increases account nonce Responding with "success" keeps transaction in "pending" state for metamask and users have to know about "account nonce" to manually set it for next transaction. I want ability to signal to MetaMask that transaction was "Dropped" without increasing account nonce.@mevalphaleak were you able to find a solution for signaling a "dropped" transaction without increasing the nonce?
I suppose the RPC could return a nonce+1, when metamask marks the transaction as "dropped" the RPC can resume returning the correct nonce.
Curious what your solution is.
Returning nonce+1 can mess up lots of things accidentally. My endpoint returns nonce of 1 billion(1E9) to drop stuck transactions in MetaMask and it assumes that no-one would ever make 1 billion txs on single Ethereum address which is pretty reasonable assumption.
for the bundle monitoring have a RESTful API that users can query with their account address to check status, it can be integrated into something like blocknative's notify.js library, this is helpful especially if you are connecting to multiple different RPC endpoints that are not public.
But how will this help with "discarded" transactions and use case I've explained above? Yes it's easy to provide status of bundle and respond that it's "discarded" the problem that there's no such status in MetaMask. Responding with "failed"(via
eth_getTransactionReceipt
) always increases account nonce Responding with "success" keeps transaction in "pending" state for metamask and users have to know about "account nonce" to manually set it for next transaction. I want ability to signal to MetaMask that transaction was "Dropped" without increasing account nonce.@mevalphaleak were you able to find a solution for signaling a "dropped" transaction without increasing the nonce? I suppose the RPC could return a nonce+1, when metamask marks the transaction as "dropped" the RPC can resume returning the correct nonce. Curious what your solution is.
Returning nonce+1 can mess up lots of things accidentally. My endpoint returns nonce of 1 billion(1E9) to drop stuck transactions in MetaMask and it assumes that no-one would ever make 1 billion txs on single Ethereum address which is pretty reasonable assumption.
How does this respond if you have, for example, a few queued transactions and the first one in line is dropped?
for the bundle monitoring have a RESTful API that users can query with their account address to check status, it can be integrated into something like blocknative's notify.js library, this is helpful especially if you are connecting to multiple different RPC endpoints that are not public.
But how will this help with "discarded" transactions and use case I've explained above? Yes it's easy to provide status of bundle and respond that it's "discarded" the problem that there's no such status in MetaMask. Responding with "failed"(via
eth_getTransactionReceipt
) always increases account nonce Responding with "success" keeps transaction in "pending" state for metamask and users have to know about "account nonce" to manually set it for next transaction. I want ability to signal to MetaMask that transaction was "Dropped" without increasing account nonce.@mevalphaleak were you able to find a solution for signaling a "dropped" transaction without increasing the nonce? I suppose the RPC could return a nonce+1, when metamask marks the transaction as "dropped" the RPC can resume returning the correct nonce. Curious what your solution is.
Returning nonce+1 can mess up lots of things accidentally. My endpoint returns nonce of 1 billion(1E9) to drop stuck transactions in MetaMask and it assumes that no-one would ever make 1 billion txs on single Ethereum address which is pretty reasonable assumption.
How does this respond if you have, for example, a few queued transactions and the first one in line is dropped?
It doesn't allow to send "skip"(nonce+1) transactions if currently pending transaction was routed to "MEV relays". Pretty much endpoint accepts several transactions at once from same user only if all previous txs were routed publicly. (Endpoint routes like 90% of transactions to public mem-pool and they don't need any MEV protection).
I think this is pretty viable, we're looking at it as a plausible Q3 priority right now.
Any updates on this. We're now well into Q4, with no PR's even opened for this particular feature.
Edit : Optionally, if anyone can direct me to a browser extension wallet that does offer this functionality, that would also be great. I don't want to stall the development of my dApp if not necessary.
t a good look and neither is it a good
I can speak for most of the people who replied here to say that we are actually very well aware of this, and it's how we are currently doing things. But it's a workaround and it's not a good look and neither is it a good approach.
Could you expand on this please. Maybe with some sample code ?
Any update on this? Q3 has passed and even more use-cases are being discovered for manually bringing your own transactions to the chain.
Nonce management is a concern, but all of the above already create those issues and metamask should employ more defensive mechanisms for dealing with nonce anyway. People using the above [advanced] use-cases are likely familiar with reset-nonce functionality in metamask.
Additionally, if we could sign transactions using this new function from accounts that don't [yet] have ETH, it would open up even more possibilities.
I think this is pretty viable, we're looking at it as a plausible Q3 priority right now.
Has there been any progress on this, Q3 is long gone
still an issue, and even more users complaining now since the recent infura outage
https://github.com/MetaMask/metamask-extension/issues/13249
Any update on this? Q3 has passed and even more use-cases are being discovered for manually bringing your own transactions to the chain.
- Using private tx pools (Flashbots protect, etc)
- Gasless TX
- Bundling multiple tx together
- whitehat
Nonce management is a concern, but all of the above already create those issues and metamask should employ more defensive mechanisms for dealing with nonce anyway. People using the above [advanced] use-cases are likely familiar with reset-nonce functionality in metamask.
Additionally, if we could sign transactions using this new function from accounts that don't [yet] have ETH, it would open up even more possibilities.
@epheph you would be interested in this conversation here ser https://ethereum-magicians.org/t/eip-3361-eth-signmessage/5969/
@danfinlay with all the stuff going around managing custom networks for metamasks next release it seems like this is the best time to do this
referencing these issues: Unable to add additional RPCs for a given chainid #13249 https://github.com/MetaMask/metamask-extension/issues/13249
Create a new screen/page: "Add network details/confirmation" #12899 #13106 https://github.com/MetaMask/metamask-extension/pull/13106
Create a new feature: "Add custom network" (Add popular networks integration) #12903 https://github.com/MetaMask/metamask-extension/issues/12903
Metamask is currently re-implementing the way Infura is integrated to accommodate this now
Metamask is currently re-implementing the way Infura is integrated to accommodate this now
Metamasks latest update that was suppose to re-implement the way Infura is integrated has accidentally banned any rpc provider from working
Metamasks latest update that was suppose to re-implement the way Infura is integrated has accidentally banned any rpc provider from working
I checked, this isn't accurate, and it's also off topic of this thread. Please create new issues if you have any unrelated topics to raise.
Two years later and this is even more of an issue now than ever. With proposer-builder separation, transaction relays have become commonplace and Dapps wish to protect their users from MEV, but without a way to have a transaction be signed without broadcast, we have no way to send these into relays without changing a users chain, which is global.
I/my org would be willing to put a bounty on a solution to this. As I understand it, the current issue is that Metamask explicitly disables support for the eth_signTransaction RPC method. The reasons were to do with nonce handling as of 2019-ish I believe, from #3475.
There is now a permissions feature in Metamask that would allow users to agree to sign without submitting txns - and as long as the UI explained correctly what was occurring, this would be okay. As far as nonces go, the worst case would be a user clobbered their own txn that was signed but not submitted with a tx that was signed and submitted by MM by not caching the nonce variable long - which is up to the dApp to handle.
It is now official: https://support.metamask.io/transactions-and-gas/transactions/smart-transactions
Please note that configuring a custom Ethereum RPC within your network settings will override Smart Transactions.
We’re able to offer this capability by auctioning transactions to a network of trusted block builders, who may bid for the opportunity to include them in a block. Block builders can earn money from some transactions by backrunning them
Pretty evident why this will never be enabled comrades
@sambacha Looking forward to hearing from the MM team on whether this was purely a business decision as you suggest.
Currently, MetaMask has 2 huge flaws which block the development of gasless tech and front-running protection for users:
1. The inability to sign transactions without also broadcasting them (first raised in #3475).
Relays which protect users from front-running require bundles of signed transaction to be submitted to them (ie flashbots which has 60% of network hashpower right now). Unlike other wallet providers, MetaMask refuses to just sign a transaction without also attempting broadcast. I understand the reasoning for this at the time, but the situation on Ethereum network is now very different from before, and this is a common user requirement. A huge % of blocks now contain bundle-style transactions, and there are a bunch of white-hat tools being built around this need (https://flashbots.tools/)
2. Exception thrown by MetaMask when submitting 0 GWEI transactions A potential workaround for 1 is to connect MetaMask to a custom backend RPC to capture the serialised transaction and increment nonces for multiple user tx's in a bundle. However, MetaMask actually throws exceptions on any TX prices under 1 GWEI. This issue makes it impossible for gasless applications to support MetaMask.
Unfortunately, as part of a team working on next-generation gasless technology, we are in the difficult situation of having to recommend our users away from MetaMask, and instead onto other wallet providers which don't have this limitation. This is a tough move to make considering the popularity of MetaMask and our personal support for the project, but unfortunately in its current form MetaMask architecture blocks support for front-running protection and gasless transactions.
As the situation on Ethereum is different to when decisions such as #3475 were taken, can we revisit this issue and see if there is appetite in the MetaMask team for change. There are a lot of projects and teams now building in the gasless/privacy sphere, and without the issues outlined above being addressed there is a risk of MetaMask no longer being the de-facto wallet for next-generation applications.