horizontalsystems / unstoppable-wallet-android

A powerful non-custodial multi-wallet for Bitcoin, Ethereum, Binance Smart Chain, Avalanche, Solana and other blockchains. Non-custodial crypto and NFT storage, onchain decentralized exchange, institutional grade analytics for cryptcurrency and NFT markets, extensive privacy controls and human oriented design. Implemented on Kotlin.
https://unstoppable.money
MIT License
826 stars 356 forks source link

Allow sending Ethereum transaction with lower gas fee than current block's fee + custom nonce #5673

Closed aleqx closed 1 year ago

aleqx commented 1 year ago

It's not only very convenient, but also much cheaper to set a lower gas fee and send it to the mempool and then mind my own business while the pending tx in the mempool waits for the blockchain's base fee to drop. This is basic stuff, and is how one can get cheap transactions. Many times I'm not in a hurry.

Completely preventing the above goes against blockchain core principles, really.

Can you please allow this - with a warning, etc - rather than prevent it?

Also, can we please have a an advanced setting for setting a custom nonce? Cancelling/replacing transactions is sometimes needed if the base fee climbed and doesn't come back down, as that tx then prevents any subsequent transaction from being confirmed.

esengulov commented 1 year ago

hi, to our knowledge when the fee is lower than base fee the transaction is dropped right a way and doesn't stay pending in the mempool. As for the Nonce, we are looking into adding this option, expect it soonish.

esengulov commented 1 year ago

Also wanted to say that the option to change nonce is already present for 'pending transactions'. When transaction is pending the user has an option to speedup or cancel it. Perhaps you're looking for an additional nonce settings in transaction send screen?

aleqx commented 1 year ago

hi, to our knowledge when the fee is lower than base fee the transaction is dropped right a way and doesn't stay pending in the mempool.

I'm afraid that's entirely false (I'm curious where did you get that information from?). Unless you are using a badly configured node, nodes accept transactions with lower base fee just fine. Have you ever looked at the mempool of a node? Here's an example of a tx submitted with a base fee of 2 gwei that has been sitting in the mempool for 3 days already: https://i.imgur.com/enRzGLU.png ... plenty other examples in https://etherscan.io/txsPending (see gasPrice column). Even transactions with 0 gwei are accepted (sort by gasPrice column)

You can also test it yourself using Metamask or any other client that doesn't prevent it. For the past month I have been using 10 gwei, sometimes sat in the mempool for a few hours.

This is the standard way to interact with the chain without overpaying. The same goes for bitcoin.

UW is currently preventing a very basic functionality.

p.s. It's also theoretically false if you read the yellow paper and eip-1559. The base fee of the next block is set by the chain according to block gas utilization of the last two blocks -- it does not depend on the base fee (well, actually max fee) set by the users on each transaction. It decreases in the next block if the current block gas is below the hardcoded 15m gas. Consequently, by design, transactions with lower max fee than the current block base fee are accepted as there is always non-zero probability the block base fee will drop. Depending on the node software, node operators may adjust some parameters, e.g. mempool timeout of pending transactions, but as you can see in my links above, the defaults are very permissive, and rightly so.

Perhaps you're looking for an additional nonce settings in transaction send screen?

Yes, that's what I suggested

aleqx commented 1 year ago

Disappointing to see this closed as completed. You may at least want to read the above reply

omurovch commented 1 year ago

Hi @aleqx,

Yes you're right, ethereum protocol allows sending transaction with fee lower than base fee. Basically, UW disallows this because of two things:

1) To prevent stuck transactions. Such transaction will be stuck in mempool till base fee drops. User might not be aware of eip1559's complicated fee logic and send transaction with low fee in order to save on fee. But since transaction will be stuck most probably, user would have to pay more fee to speed up the transaction.

2) Some nodes may not accept (deliberately or not) transactions with fee < base fee. For instance, RPC provider Infura returns with error "max fee per gas less than block base fee" for gas limit estimation request.

We are planning to add more RPC providers. We may allow sending with low fee if those providers don't return an error like Infura.

What do you think what should be a minimum allowable base fee? Because setting base fee 1 wei is somewhat meaningless.

Also regarding nonce settings, can you please eloborate what are your use cases where you need ability to change transaction nonce?

aleqx commented 1 year ago

First, I should say I like UW so far, and would like to donate if I continue to use it -- hopefully I can stop carrying Metamask around with me (see more below). I hope you guys can stick to both healthy principles and full open source (i.e. hope you can find a sustainable income revenue stream and can stick to your guns; you may recall that Coinomi also started as open source then turned closed source, claiming bad actors providing copies with backdoors and keyloggers).

UW disallows this because of two things:

Actually, those aren't reasons to disallow it, even if they were both true (2nd is largely untrue). They are just reasons to not make it a default. Have the default to a healthy value, but give the option under an advanced setting and show a warning when used. Not having an option means losing adoption of your software in favor of other, better, more feature rich software.

It's not true that Infura doesn't accept base fee lower than current block's fee. Metamask uses Infura by default (rpc https://mainnet.infura.io/v3/) and it allows it just fine, it always did. In fact, I myself have yet to meet a provider that doesn't. Most are owned by infura anyway so it's hardly an issue ... but even if it's rejected, the rpc error will be reflected to the users and the user will know what to do (no harm done as no gas is used).

What do you think what should be a minimum allowable base fee?

I actually disagree with the question :). As a software developer myself, I disagree with imposing hard limitations (that's how my software loses adoption). As I said above, I think it's always best to have healthy defaults for laymen users, but give the option under an "advanced" setting, and then a visible warning about the consequences. Leave the choice to the user. Again, Metamask does this well, so does Rabby and many other other wallets. To answer your question: let the user go down to 0 if they want to, a warning is enough.

You also have the benefit that crypto users are far more savvy compared to, say, traditional banking users where you have to worry about the UX for 90 year old grandmas using your app. We web3 devs are in fact encouraged to provide advanced options, instead of excluding them, because there is demand from the current audience. 20 years from now will be a different story.

Also regarding nonce settings, can you please eloborate

In ethereum, setting the nonce of a tx equal to a pending tx nonce is how you can replace that tx (as long as it has a sufficiently higher fee, etc etc, to avoid having the original tx confirmed before the new one). I may want to change the amount of the tx because I changed my mind, or I may want to use a different dex because i changed my mind. Both of this happened this week in fact, e.g. realized I should have provided more collateral for an Aave loan, so I had to initiate a new tx from Aave to supply collateral.

It's not always just "cancel" or "speed up". Sometimes you do need to set a custom nonce for a new tx. The nonce allows me to issue a completely new tx to override any pending one, they don't have to be of the same kind. Hate to say it again, but Metamask does this well. Surely you used Metamask before? By default show no custom nonce field (again: healthy defaults for laymen users), but have an option in settings to enable custom nonce for every transaction so that I can change the default nonce (equal to 1+last confirmed tx nonce) to any nonce value to replace a pending tx. Advanced functionality for advanced users ... there are many of us out there. It would allow me to stop carrying Metamask around.

p.s. I have more suggestions to improve security and UX, e.g. please blank the screen in the android task manager, else someone who steals my phone can see the wallet amounts in the task manager. I can post them as separate github issues. Another is more asse4ts on different chains to recognize their USD value (e.g. Nexo on Polygon currently doesn't have a USD value as UW doesn't recognize it natively)

p.p.s. Even 1 gwei may get confirmed. Unlikely right now, but not zero probability later -- we don't know yet how L2 activity will evolve, which may free up L1 block space. 1 gwei or even lower was the norm not that long ago (2017-2019), then 100-200 gwei was the norm 1-2 years ago, now 10-20 gwei being the norm now owing to more L2 adoption. Theoretically, even 0 gwei can get included if the mempool gets close to empty.

esengulov commented 1 year ago

I hope you guys can stick to both healthy principles and full open source (i.e. hope you can find a sustainable income revenue stream and can stick to your guns; you may recall that Coinomi also started as open source then turned closed source, claiming bad actors providing copies with backdoors and keyloggers).

:) no plans to go to closed source.

Actually, those aren't reasons to disallow it, even if they were both true (2nd is largely untrue). They are just reasons to not make it a default. Have the default to a healthy value, but give the option under an advanced setting and show a warning when used. Not having an option means losing adoption of your software in favor of other, better, more feature rich software.

We are going to add it. Likely in the release 0.31. You may follow the progress on it here. The UI is already in the works.

p.s. I have more suggestions to improve security and UX, e.g. please blank the screen in the android task manager, else someone who steals my phone can see the wallet amounts in the task manager. I can post them as separate github issues. Another is more asse4ts on different chains to recognize their USD value (e.g. Nexo on Polygon currently doesn't have a USD value as UW doesn't recognize it natively)

Please do post them as separate issues.

Thank you for insightful feedback!

TalgatNurdinov commented 1 year ago

Nonce input in the fee settings page

https://www.figma.com/file/M9MoqHabdQCdu2a9pIEo5N/v.-2.0--Unstoppable-Design-System?node-id=66821%3A290320&t=e7OZ7jSvLKM31ndy-11

aleqx commented 1 year ago

Nice!

It would be nice if you also allowed entering numeric values for the base fee and tip (users may want more granularity than then + and - buttons currently offer)

aleqx commented 1 year ago

I would move the nonce field at the bottom though, rather than the top (and add a small warning text like "This is an advanced field, if you don't know what it means then do not change it"), to avoid confusing beginners.

Also, it should be "Tip" instead of "Max tip" if you choose to use "Max base fee" (which is in fact unusual) (^^^ see reply below, I don't encourage this even if it's theoretically valid)

aleqx commented 1 year ago

Also, it should be "Tip" instead of "Max tip" if you choose to use "Max base fee" (which is in fact unusual)

I believe UW has a display bug that is misleading users - you most likely mean "max total fee" instead of max base fee.

To clarify: The protocol settings that are actually sent to the chain are max total fee (or max fee) and max priority fee (also the most common GUI options in wallets too, e.g. Metamask). There is no such thing as max *base* fee in eip-1559, I'm assuming that's just a UW display bug, but misleading nonetheless as it affects fee computation for users.

The priority fee is usually spent partially whenever the user's max priority fee plus the chain's base fee is less than the user's max total fee -- this almost always happens. You always spend MIN(max_fee_per_gas, (base_fee_per_gas + max_priority_fee_per_gas))

UW right now uses max *base* fee and also max priority fee which actually make it impossible to deterministically compute both the max total fee and max priority fee that are sent to the chain (hence why I'm assuming it's a display bug). Theoretically, you could compute them if you used max *base* fee and *fixed* priority fee (my previous post) but it's a little tricky in practice to ensure the chain spends your priority fee fully (similar to how it's less trivial in eip-1559 to spend all ETH in a wallet without any remaining dust), which is why I crossed out my previous post suggestion.

Best to just change max base fee to max total fee, to match other Ethereum wallet software.

omurovch commented 1 year ago

hi @aleqx, all your points will be taken into account. Thanks!

Dianaaiym commented 1 year ago

Texts info

aleqx commented 1 year ago

@Dianaaiym I'm not sure what those are supposed to be? Not all of them are technically correct either

omurovch commented 1 year ago

@aleqx can you provide your descriptions? May be you can come up with better ones. The aim is to make it concise and not too technical.

esengulov commented 1 year ago

@Dianaaiym I'm not sure what those are supposed to be? Not all of them are technically correct either

Thanks for the input, we tried to make sure that descriptions are understandable to the person who might not be familiar with the EIP1559 terminology. After all, someone who is already familiar with the EIP won't need these descriptions.

At the same time we take into account that the strings are going to be converted into other languages as well.

How would you rephrase those? Your feedback would be greatly appreciated

aleqx commented 1 year ago

I know they are supposed to be definitions/explanations of technical concepts, I just don't understand what the exact purpose is - do you intend to put them into your help section on your website, or in the help of the app, or in the Send panel of the app, or ...? They seemed too long to be strings in the Send panel, but perhaps you want them in the tooltips of each item in the Send panel.

I rephrased and corrected some for you below. They are more verbose than yours but feel free to cut down on text if you wish.

You could drop the word "rate" from all the items, to keep it more inline with other documentation, although "rate" is correct as it refers to "fee per gas". Official EIP also uses "fee" in text, but "fee_per_gas" in equations and code: https://github.com/ethereum/EIPs/blob/master/EIPS/eip-1559.md

Max Estimated Transaction Fee

The maximum fee to be paid for this transaction, computed based on the gas limit, max fee rate and max priority fee values below. The actual fee paid will normally be lower.

Gas Limit

Transaction complexity is measured in units called "gas". It varies depending on the smart contract being executed. The Gas Limit is the estimated maximum gas needed to execute it. The actual gas used will normally be lower.

Base Fee Rate (Gwei)

The network protocol determines the base price per gas for each block, called base fee rate. It varies according to the network utilization level from block to block. It can increase or decrease by no more than 12.5% in the next block, making fees more predictable. The value shown here is the current block's base fee rate.

Max Miner Priority Fee Rate (Gwei)

Users pay priority fees to incentivize a transaction to be confirmed more quickly. They are sometimes called tips. The max priority fee rate is the maximum additional price per gas the user is willing to pay on top of the base fee rate. The value shown here is suggested based on predicted network conditions. The actual priority fee will normally be lower. Setting this to zero may result in a long waiting time for transaction to be confirmed, as it is placed at the end of the pending transactions queue from all users.

Max Fee Rate (Gwei)

This is the maximum total price per gas the user is willing to pay. It must cover the network's base fee rate and max priority fee rate. The value shown here is suggested based on an estimate of the next block's base fee rate plus the max priority fee rate chosen by the user. The actual fee rate paid will normally be lower. Setting this lower than the current base fee rate will limit the fee paid, but will result in longer waiting times for the transaction to be confirmed, or even in a stuck transaction.

Transaction nonce

The nonce is a unique integer value for a transaction within the user's wallet. It normally increments with each submitted transaction and does not need changing. Advanced users can set it equal to a nonce of a pending transaction in order to cancel and replace that transaction, as long as the new transaction has a sufficiently higher fee to prevent the old one from being confirmed instead (for example, they may want to speed up its confirmation, or to change transaction parameters entirely). When multiple pending transactions have the same nonce, only one gets confirmed, typically the one with the highest fee.

p.s. The nonce can be set to any value. It does not have to be +1 or equal to a pending transactions. You can set it to +55 if you wish, or if you have 10 pending transactions, each with different nonces, you can set it to any of the 10 values. Wallets choose to do last_nonce+1 because it's the simplest strategy to get unique integers, but in fact they should do max(nonces)+1 or else you may get a nonce collision and the node will reject it.

p.p.s. Don't use the word "miner" any more. Most networks are PoS now, which use "validator". I avoided the use of either miner or validator to make it generic

esengulov commented 1 year ago

@aleqx thanks a ton, your descriptions do look better :) We are going to adapt them. These are meant for tooltips in Send panel. See the image below for UI screen.

@omurovch @kansalt @TalgatNurdinov @esen @Dianaaiym

Please note updated tooltips description texts and titles

Max Fee

The maximum estimated transaction fee to be paid for this transaction, computed based on the gas limit, max fee rate and max priority fee values below. The actual fee paid will normally be lower.

Gas Limit

Transaction complexity is measured in units called "gas". It varies depending on the smart contract being executed. The Gas Limit is the estimated maximum gas needed to execute it. The actual gas used will normally be lower.

Base Fee

The network protocol determines the base price per gas for each block, called base fee rate. It varies according to the network utilization level from block to block. It can increase or decrease by no more than 12.5% in the next block, making fees more predictable. The value shown here is the current block's base fee rate.

Max Priority Fee

Users pay priority fees to incentivize a transaction to be confirmed more quickly. They are sometimes called tips. The max priority fee rate is the maximum additional price per gas the user is willing to pay on top of the base fee rate. The value shown here is suggested based on predicted network conditions. The actual priority fee will normally be lower. Setting this to zero may result in a long waiting time for transaction to be confirmed, as it is placed at the end of the pending transactions queue from all users.

Max Fee Rate

This is the maximum total price per gas the user is willing to pay. It must cover the network's base fee rate and max priority fee rate. The value shown here is suggested based on an estimate of the next block's base fee rate plus the max priority fee rate chosen by the user. The actual fee rate paid will normally be lower. Setting this lower than the current base fee rate will limit the fee paid, but will result in longer waiting times for the transaction to be confirmed, or even in a stuck transaction.

Transaction nonce

The nonce is a unique integer value for a transaction within the user's wallet. It normally increments with each submitted transaction and does not need changing. Advanced users can set it equal to a nonce of a pending transaction in order to cancel and replace that transaction, as long as the new transaction has a sufficiently higher fee to prevent the old one from being confirmed instead (for example, they may want to speed up its confirmation, or to change transaction parameters entirely). When multiple pending transactions have the same nonce, only one gets confirmed, typically the one with the highest fee.

send

aleqx commented 1 year ago

These are meant for tooltips in Send panel.

In that case, if I were you I would leave them longer and clearer, rather than succinct but vague or lacking. Being tooltips it doesn't affect the interface, but users would appreciate both actually understanding what's going on and being educated in an easy non-technical way.

Could you also allow inputting numeric values, and not just use +/- buttons?

TalgatNurdinov commented 1 year ago

Could you also allow inputting numeric values, and not just use +/- buttons?

Yes, input fields allow manual input from the numpad