near / NEPs

The Near Enhancement Proposals repository
https://nomicon.io
214 stars 137 forks source link

Proposal for web3-compatible wallets support #518

Open alexauroradev opened 11 months ago

alexauroradev commented 11 months ago

Summary

This proposal introduces a significant enhancement to the NEAR Protocol, aiming to make its ecosystem closer with Ethereum by integrating Web3 wallet support. The primary goal is to resolve the incompatibility issues between Ethereum and NEAR standards, thus facilitating a seamless user experience for Ethereum users on the NEAR platform. Central to this initiative are several components: the Wallet Contract, Wallet Selector Extension, RPC-Translator, Transaction Relayer, and the Ethereum Translation Contract. Together, these elements enable Ethereum-compatible transactions within NEAR, ensuring that users familiar with Ethereum's environment face minimal friction in using NEAR dApps.

Key features include a protocol change that embeds the Wallet Contract for implicit Ethereum-style accounts as a fundamental feature of NEAR and simulating Ethereum standards for transaction compatibility on the level of this contract.

Proposal also acknowledges inherent limitations and risks, which, however, are manageable. The proposal outlines future opportunities for extending its capabilities to include more Ethereum standards and exploring further integrations. The project's scope is vast, requiring rigorous development, testing, and collaboration across various components and entities of the NEAR ecosystem.

In essence, this proposal strives to enhance NEAR's interoperability with Ethereum, significantly improving accessibility and user experience for a broader audience in the blockchain community.

The proposal has core and advanced topics. Pieces that are marked with [COMPLEX] may be omitted during the first read.

Problem Statement

Currently, the Ethereum ecosystem is a leading force in the smart contract blockchain space, boasting a large user base and extensive installations of Ethereum-compatible tooling and wallets. However, a significant challenge arises due to the incompatibility of these tools and wallets with NEAR Protocol. This incompatibility necessitates a complete onboarding process for users to interact with NEAR contracts and accounts, leading to confusion, decreased adoption, and the marginalization of NEAR Protocol.

Implementing Web3 wallet support in NEAR Protocol, with an emphasis on user experience continuity, would significantly benefit the entire NEAR Ecosystem.

Goals / Deliverables

The primary goal is to develop a solution enabling Web3 wallet users to seamlessly interact with NEAR Protocol while retaining their user experience with other EVM-compatible networks. This solution should be straightforward, requiring minimal changes to the NEAR protocol, avoiding potential reversals of protocol changes, and minimizing the need for extensive user education. Additionally, it should prioritize minimizing the risk of phishing attacks that trick users into signing indecipherable data. It's crucial that transactions signed by users are processed unambiguously at the NEAR blockchain level, eliminating the need for a trusted intermediary.

Prior work

Previous efforts have been made in this area. Notable examples include the nEth project and the Metamask Snap for NEAR. These projects enable Metamask compatibility with NEAR natively but fall short in supporting arbitrary Ethereum wallets. Moreover, neither project provides a user experience comparable to that of EVM-compatible chains.

With nEth, users must sign EIP-712 messages, which differs from executing standard transactions in Metamask. The Metamask Snap project effectively creates a distinct application within Metamask, separate from typical Metamask workflows. This application cannot interact with Metamask keys, nor can it display token balances or transaction histories, and it lacks integration with Ledger. Both projects also necessitate funding new accounts from external sources.

Aurora XCC offers another avenue for Ethereum wallet users to interact with Near native. However, this approach has drawbacks, such as the requirement for WNEAR (NEP-141 wrapped NEAR bridged into Aurora from Near native) for many applications. This necessity complicates users' understanding of transaction costs. Additionally, XCC introduces a gas overhead compared to direct Near usage, potentially hindering user interactions with some applications due to Near's 300 Tgas transaction limit.

Technical Description

Ethereum and NEAR exhibit several fundamental incompatibilities that impact areas such as transaction formats, signature algorithms, addressing schemes, public RPC APIs, and interaction workflows. This proposal seeks to effectively conceal and/or resolve these incompatibilities to facilitate standard wallet operations like balance inquiries, account top-ups, transfers of fungible tokens, and smart contract function calls on the NEAR blockchain.

Solution Overview

The proposed solution comprises five key components:

  1. Wallet Contract (WC): An on-chain smart contract designed to receive, validate, and execute Ethereum-compatible transactions on the NEAR blockchain. It functions as a user account.

  2. Wallet Selector Extension: A frontend module that connects NEAR-compatible wallets to dApps. This project aims to develop add-ons supporting Ethereum wallets, primarily to create Ethereum-compatible transactions from NEAR-compatible inputs.

  3. RPC-Translator (RPCT): Given that Ethereum wallets access blockchain state via a specific Web3 API, the RPC-Translator is designed to provide Ethereum methods implementations using the NEAR RPC as a data source. This component, along with the Transaction Relayer, is publicly hosted and accessible to all users. While the RPC-Translator and Transaction Relayer are operated together, they are listed separately to highlight their distinct functions.

  4. Transaction Relayer (TR): Ethereum wallets cannot generate NEAR-compatible transactions. Instead, Ethereum-compatible transactions produced by them are processed by the Transaction Relayer, which embeds it into a NEAR transaction and forwards it to the user’s Wallet Contract. Again, while operated in conjunction with the RPC-Translator, the Transaction Relayer is distinct in its role.

  5. [COMPLEX] Ethereum Translation Contract (ETC): Addressing the incompatibility between NEAR's human-readable account names and Ethereum's cryptographic hash-based addresses, the ETC functions as an on-chain mapping system. This system records NEAR-compatible input values (like NEAR account names and smart contract function names) and maps them to their corresponding Ethereum-compatible cryptographic hashes. This feature is vital for preserving familiar user experiences, such as recognizing ft_transfer operations in NEP-141 as fungible token (ERC20) transfers, rather than generic contract calls, and ensuring that fungible token balances are displayed in Web3 wallets.

Transaction Flow

The transaction flow between components is outlined as follows:

Below are detailed examples, with implementation specifics provided subsequently.

Example Transaction Flow: Incoming $NEAR Transfer

This process aligns with the user experience expectations of both NEAR and Ethereum users.

Example Transaction Flow: Outgoing Function Call

This method meets the expectations of both NEAR developers and Ethereum users.

[COMPLEX] Example Transaction Flow: NEP-141 Transfer from Metamask

This transaction flow adheres to the expectations of Ethereum users.

Wallet Selector and Transaction Transformation

Ethereum transactions, RLP encoded, encapsulate the following information:

For function calls to a smart contract, the data field contains:

Both To and From addresses, encoded in hexadecimal and prefixed with “0x”, comprise 20 bytes. The From address is the right-most 20 bytes of the Keccak-256 hash of the binary public key of the transaction sender (an EOA, externally owned address). The To address signifies either the recipient EOA or an on-chain contract address.

The Function Selector is the first 4 bytes of the Keccak-256 hash of the full function signature intended for the contract specified by the To address. For instance, the ERC-20 standard function transfer(address _to, uint256 _value) has the Function Selector keccak-256("transfer(address,uint256)")[0,4], equal to 0xa9059cbb.

Proposed Ethereum Transaction Construction

Special Considerations for To and Data Field

The To address and data field creation vary based on the dApp's input to the Wallet Selector:

For the data field:

Wallet Contract

The Wallet Contract is designed to process Ethereum transactions through the method executeRLP(target: AccountId, rlp_transaction: Vec<u8>). The following steps outline its operation:

  1. Parse the Transaction: It starts by parsing the incoming Ethereum transaction.

  2. Verify Target Address: It checks if the target equals the transaction's To address, or if keccak256(target)[12,32] matches the To address.

  3. Signature Verification and Extraction: The contract verifies the signature to extract the Public key and chainID, ensuring the integrity of the transaction.

  4. From Address Creation and Verification: It generates the From address from the public key and confirms that it aligns with the current_account_id (the ID of the Wallet Contract).

  5. ChainID Validation: The contract validates that the chainID corresponds to the constant defined for NEAR.

  6. Nonce Verification and Update: It ensures the transaction nonce matches the stored nonce and then increments the stored nonce by one.

  7. [COMPLEX] Value Calculation: The contract sets the Value by multiplying the Ethereum transaction value by 1e6 and adding the yoctoNear value from the data field. It then confirms the attached deposit is either greater than or equal to this value or identifies it as a self-transaction (refer to TR and Gas Payment section for details).

  8. [COMPLEX] ETC Lookup and Execution for Ethereum Transfers:

    • If the To Address equals the target and the value is more than zero (indicating an Ethereum transfer), the Wallet Contract consults the ETC ("Ethereum Translation Contract") for the To address.
      • If the ETC lacks an entry for the To address, it implies the recipient is an Ethereum EOA. In this case, the transaction is an ordinary transfer and should be executed as such.
      • If the ETC contains an entry for the To address, indicating the recipient is a NEAR smart contract simulating an Ethereum standard (like NEP-141), the data field of the transaction is parsed against supported ERC standard calls to form the appropriate NEAR action.
  9. NEAR Native Account Transfer: If the data field is empty, it transfers the value to the target, typical for a $NEAR transfer to a NEAR-native account. Values below 1e6 yoctoNEAR cannot be transferred.

  10. Function Selector and Parameter Verification: In cases where the data field is not empty, the contract verifies the function selector against matching NEAR actions.

  11. Execution of NEAR Action: After parameter verification, the Wallet Contract executes the corresponding NEAR action by creating a promise.

These steps ensure that the Wallet Contract accurately processes and executes transactions, bridging the gap between Ethereum and NEAR protocols.

Transaction Relayer and Gas Payment

The Transaction Relayer (TR) serves as an HTTP-JSONRPC endpoint compatible with Ethereum, primarily implementing the eth_sendRawTransaction method. Its validation process involves:

Upon receiving a transaction, the TR assesses the To Address:

Approaches to Gas Payment

This project's challenge lies in appropriately assigning gas costs to the Ethereum-compatible account. Our proposed solutions are:

  1. Simplified Gas Calculation: By default, all transactions use the maximum gas limit, or a limit specified by the dApp developer, to bypass the complexity of gas amount computation.

  2. Gas Payment Scheme:

    • Register the account with TR. The Ethereum wallet sets up a functionCall key on its account, authorizing the executeRLP method call with a public key from the Transaction Relayer. This setup, compatible with the Transaction Translation model, employs the AddKey Action. Received transactions under this scheme are forwarded and paid for by the Transaction Relayer. Alternatively, the Wallet Contract could reimburse the Transaction Relayer via a Transfer promise.
    • Once an Ethereum wallet is registered with the Transaction Relayer, this functionCall key is utilized for all subsequent transactions. Effectively, the Transaction Relayer operates on the account's behalf, with the account bearing all gas costs.
    • To mitigate potential abuse by the Transaction Relayer, the Wallet Contract can revoke the functionCall key in cases of non-parseable transactions, invalid signatures, or nonce replay (subject to specific time and value deltas). This limitation restricts a dishonest Transaction Relayer to a single invalid transaction's NEAR gas cost (typically not exceeding 0.03 $NEAR under normal conditions).

[COMPLEX]: This approach avoids unnecessary cross-contract calls, maintaining NEAR protocol interactivity for Ethereum users.

Optional Free Transactions: Some relayers may offer a number of free transactions as an incentive for users to engage with NEAR. This can be achieved by the relayer's account directly calling the Wallet Contract's executeRLP function, instead of using the user’s account's functionCall access key. Some of these transactions may require attachment of $NEAR for covering storage staking or other requirements of NEAR protocol standards.

RPC Translator

The RPC-Translator, functioning as an HTTP-JSONRPC endpoint compatible with Ethereum, implements a set of methods crucial for supporting Ethereum wallets. These include:

These methods enable Ethereum wallets to create transactions and access transaction status. The proposal also accommodates direct $NEAR and ERC20 transfers from Ethereum wallets.

[COMPLEX] Ethereum Standards Simulation and Ethereum Translation Contract

To bridge the gap between NEAR's and Ethereum's differing addressing schemes, especially for interactions initiated directly from web3 wallets, the Ethereum Translation Contract (ETC) is pivotal. It maintains a simple mapping between Ethereum transaction hashes and corresponding NEAR account names:

The ETC provides two core functions:

NEAR accounts intending to interact with Ethereum wallets must register with the ETC. This registration is permissionless, allowing third-party on-the-fly registrations if necessary.

It's important to note that ETC registration is not mandatory for most NEAR applications. For instance, applications like ref.finance, typically not called directly from Metamask, would route their contract calls through the frontend, thereby utilizing the Wallet Selector's transaction translation.

In the initial phase, the focus will be on $NEAR token transfers and ERC-20 transfers, and approval management in the Ethereum Standards Simulation. Future expansions may include additional standards in the Wallet Contract (refer to the testing and upgrades section). The ETC itself does not require upgrades to support these enhancements.

Implementation Notes

The proposed solution necessitates a protocol change for deploying the Wallet Contract on implicit Ethereum accounts. To facilitate this, we suggest integrating the Wallet Contract as a core feature of the NEAR protocol. This change would treat all Ethereum accounts as if they already have the Wallet Contract deployed.

Additionally, to enhance security and align with Ethereum's operational flows, we propose prohibiting the deployment of other contracts on Ethereum accounts. This can be achieved by restricting Ethereum users from adding full access keys to these accounts, thereby significantly reducing the attack surface.

Testing and Upgrades

Integrating the Wallet Contract as a protocol feature requires synchronized testing and upgrade processes between nearcore and the Wallet Contract. Fortunately, due to the absence of cross-dependencies, the Wallet Contract can be developed, maintained, and tested independently. Its latest stable version would then be incorporated into the nearcore Testnet release, followed by integration into the Mainnet release.

We recommend that any modifications to the Wallet Contract's functionality, such as adding support for new Ethereum standards over time, should be incorporated into the NEAR enhancement process. This approach ensures a structured and systematic update mechanism.

The development and testing of off-chain components, like the RPC-Translator, will precede those of on-chain components. This sequential approach allows for thorough testing and refinement of each component before integration into the larger system.

Limitations and Risks

Future Opportunities

Scope of the Project

The scope of this project is extensive, encompassing several critical components:

  1. Core Technology:

    • Wallet Contract: A crucial, security-sensitive component requiring detailed audits and testing.
    • nearcore Protocol Upgrade: Modification to treat Ethereum-style accounts as implicit and deploy the Wallet Contract on them.
    • Wallet Selector Add-Ons: Ensuring compatibility with Metamask and WalletConnect-compatible wallets.
    • Transaction Relayer and RPC Translator: Key elements of the NEAR Web3 RPC middleware, working alongside the NEAR node.
    • Ethereum Translation Contract (ETC): A simpler component in comparison to others, yet vital for the system.
  2. Integration Testing: Given the solution's distributed nature, comprehensive integration testing is imperative.

  3. Documentation Development: Detailed documentation is necessary, enabling third-party developers to understand and potentially replicate parts of the system, except for the Wallet Contract.

  4. Node Operators Awareness and Public Infrastructure: Launching and maintaining the NEAR Web3 RPC as a public service, similar to the existing NEAR RPC, and informing third-party node operators about the update and maintenance protocols.

  5. Integration with Key Platforms: For effective Ethereum user engagement, integration with the following is essential:

    • Centralized Exchanges: Permitting withdrawals to Ethereum-style addresses.
    • Bridges: Facilitating login on the NEAR side with Ethereum-compatible wallets.
    • NEAR-native Projects: Updating Wallet Selector configurations to support Ethereum-compatible wallet logins. Anticipated challenges include wallet address display issues, requiring adjustments.
    • Products to allow for key NEAR Wallet functionality. Allowing Etherum users to exercise core features of the NEAR protcol in a dApp: staking, key management, work with lockup contracts, etc. These may be implemented as BOS components.
  6. Bug Bounty Program: Establishing a program to address potential vulnerabilities, especially for on-chain components like the Wallet Contract and ETC.

  7. Ongoing Development and Bug Fixes: The project's complexity necessitates a proactive approach to bug fixes and future enhancements.

alexauroradev commented 11 months ago

This proposal is an extension and rework of the https://github.com/near/NEPs/issues/498 Adding Ethereum implicit accounts to nearcore is managed by this tracking issue: https://github.com/near/nearcore/issues/10018

alexauroradev commented 11 months ago

Design Review Deck Design Review Call Recording

walnut-the-cat commented 11 months ago

@near/wg-protocol , please nominate SMEs who can review this NEP

mfornet commented 11 months ago

@near/wg-protocol , please nominate SMEs who can review this NEP

Notice that this is not a NEP yet but a proposal for discussion. This format is beneficial for high-level discussion, and once it becomes a NEP, we can discuss low-level details of the implementation.

Tarnadas commented 11 months ago

I just wanted to say, that I've been integrating the Aurora SDK into a Near smart contract and the asset bridging is extremely complex and also the gas usage is so high, that it's very limiting.

In theory it's a nice addition, but almost all DeFi apps would need some sort of token bridging, but all the cross contract calls consume so much gas, that you have very few gas remaining for the actual smart contract execution on Near. You can check out this example, that I co-authored and there's only 35TGas allocated for the Near smart contract call, because the remaining gas is consumed elsewhere: https://github.com/aurora-is-near/aurora-contracts-sdk/blob/ae124d2faa2cd9cfdaaeb72949610887256f122f/examples/ft-refund/solidity-contract/src/FtRefund.sol#L12

The bridged wNEAR for Aurora XCC that Alex is mentioning can be circumvented by a small code change in the Aurora SDK, but you would need to make sure, that users would have to pay for the wNEAR deposit somehow else (e.g. by depositing ETH instead, like it's done with Gas on Aurora).

Overall the Aurora SDK is somewhat impractical for DeFi apps on Near (but much better for dapps, that don't need token bridging), but it's possible to use it and you can implement some tricky workarounds to have better way of token refunding.

walnut-the-cat commented 10 months ago

@alexauroradev , please ping NEP moderator once the NEP is ready to be reviewed-

swapnilraj commented 10 months ago

I don't have a lot of context, so forgive me for that.

Is this design inspired by EIP-4337?

Ekleog-NEAR commented 10 months ago

We had some related discussion in https://near.zulipchat.com/#narrow/stream/295306-pagoda.2Fcontract-runtime/topic/Eth-wallet.20support/near/403646004 .

In particular, I think the idea of actually having some hash-containing placeholder for the smart contract code, would help us be forward-compatible with shared smart contracts. This idea would result in us having eg. near[contract hash] as the deployed code, which does not run at a risk of collision with real contracts considering these ones must start with \0asm.

Ekleog-NEAR commented 10 months ago

I’m seeing that this is suggesting keccak256(...)[12,32] as a way to generate the hashes. Why is there these [12,32] magic numbers? Is it possible to document the rationale behind this choice for the hashing function?

birchmd commented 10 months ago

I’m seeing that this is suggesting keccak256(...)[12,32] as a way to generate the hashes. Why is there these [12,32] magic numbers? Is it possible to document the rationale behind this choice for the hashing function?

On Ethereum itself the addresses are derived from the public key for the account by taking the rightmost 20 bytes (160 bits) of the keccak256 hash (this is defined in Appendix F, eq 323 of the Ethereum Yellow Paper). We are applying this same rule when deriving an address from a Near account id.

We could choose any rule for deriving a 20-byte address from a Near account id, but reusing the Ethereum rule for an Ethereum compatibility protocol is a sensible choice in my opinion (principle of least surprise for devs familiar with Ethereum).

walnut-the-cat commented 9 months ago

What's the current status of this NEP? It is ready to be reviewed by SMEs?

walnut-the-cat commented 8 months ago

another gentle ping to understand the readiness of the NEP :)

victorchimakanu commented 4 months ago

Hello everyone, I'd like to understand the current status of this NEP? Can we nominate reviewers and discuss low-level details of the implementation?

birchmd commented 4 months ago

There is an implementation prepared under a nightly feature of nearcore and it is currently being audited. The proposal also has support from the infrastructure working group (they already have a tracking issue related to tasks associated with the proposal). I suggest we move forward with the review and voting stage for the protocol change parts of this proposal (which is just the wallet contract and eth-implicit accounts).

victorchimakanu commented 4 months ago

Thank you @alexauroradev for submitting this NEP.

As a moderator, I reviewed this NEP and it meets the proposed template guidelines. I am moving this NEP to the REVIEW stage and would like to ask the @near/wg-protocol working group members to assign 2 Technical Reviewers to complete a technical review.

bowenwang1996 commented 3 months ago

As a working group member, I nominate @akhi3030 and @nagisa as SME reviewers

akhi3030 commented 3 months ago

I only see an issue here and not a PR for the NEP. Is there a link to the NEP?

birchmd commented 2 months ago

@akhi3030 I'll make a PR which contains this issue description. For now please treat the issue description as the NEP.

akhi3030 commented 2 months ago

@birchmd: thanks, I will start with the issue description. It will be great to have the PR eventually as that will make it easier to review and provide feedback.

birchmd commented 2 months ago

@akhi3030 I have created the PR: https://github.com/near/NEPs/pull/555