Contract-scoped state replication for layer 2 applications and infrastructure

[TimDaub]

Hi,

in numerous past projects, I have used web3 in combination with a full node to build a decentralized app. A repeated observation I've been able to make is the unavailability of a contract's current state. While web3 and other libraries make it easy to query for a particular piece of state in a contract, e.g. all owners of an ERC20 token, it's challenging getting a contract's full state. Here's what happens in most front ends right now:

• JS app is loaded and the web3 provider is inserted
• Hardcoded values (e.g. contract address) are used to query a third-party full node through their JSON RPC interface
• Individual state values are returned to the user. Having successfully synced all necessary state, they can start interacting with the on-chain contract.

But this leads to some problems:

• In case a developer is only retrieving the latest state, it can quickly turn outdated (e.g. leaving a page for > 1h open) and stop the user from interactions as all inputs will be deemed out-of-date by the contract. A developer will have to foresee this and continuously update the state block by block
• Most calls to the chain are reliant on the user's internet connection. Through the usage of JSON RPC, they're additionally stateful. This can make building a reliable app difficult. Offline-first approaches seem technically difficult too.
• The retrieved state is ephemeral. It could change any minute. Additionally, though, it's useless after invalidation. Why does this matter? It matters because if state would be passed on not as values but as atomic transactions (which is a technical possibility), it'd be reusable. In the sense that all transactions of a contract give its current value.
• Testing contracts in composability scenarios is really difficult. Imagine having to build a contract that interacts with a DeFi application's state. At specific values of its state, it'd trigger an action. Since having to have control over the specific state of the contract in a particular situation (e.g. an action that only triggers when the MakerDAO DAI System goes in an emergency mode) is only inaccurately replicable by e.g. retrieving all contract's storage via getStorage(address, i), testing becomes difficult too.

Lastly, there's to point out that having to rely on a full node additionally assumes either being happy with the full node provider's performance or either hosting a dedicated full node for a project. However, both of these approaches are rather unhappy as:

• full nodes now start to cost money. Dedicated full nodes also require money. Hence, if I was to try to set up a truly decentralized application I probably wouldn't be able to without entering my credit card information somewhere.
• Having to provide payment credentials additionally defeats the point of a decentralized application as it'd mean that I as the developer am responsible for keeping the application alive.

Lastly, there's the problem of some contracts not allowing the efficient retrieval of a particular state at all. Two years ago when I attempted to build a fully-decentralized ERC721 wallet I found that for some ERC721 contracts, retrieval wasn't possible as nodes stopped processing the JSON RPC call as it was too computationally intense. I later scrapped the project as here too, only a centralized approach was possible (e.g. opensea).

So by now, it should be relatively clear that building front ends that rely on full nodes is rather difficult and can hardly be achieved just using the JSON RPC endpoints. At least not while trying to build a great user experience. So what can we do to improve the situation?

Proposal: Make contract-scoped state replication possible

What is meant by contract-scoped state replication? To properly understand, I'd like you to free yourself of the current web3 architecture of Ethereum. What if instead of directly calling for a contract's state value through an ABI in web3.js, what if a front-end application was able to independently replicate the state of a contract by replaying all relevant transactions? How would this work?

Hardcoded in the frontend would be the contract's address as well as its creation transaction. From this data, a block range of transactions to replicate the contract's state would become apparent. Hence, the client would now ask the server something along the lines of:

• Give me all transactions starting from block X that have had any interaction with contract C.
• The server would now start streaming all those transactions to a client.
• On the client, a local EVM is running that can take in these transactions. For each run transaction, the local state is persistent on the client's drive somehow.
• In fact, transactions could be globally cached at the client. Transaction's content never changes and through a transaction's hash, its content can be authenticated. Hence, there could be a situation where a client e.g. already has downloaded half of all needed transactions when using e.g. 1inch as half of them are also manipulating the state of e.g. uniswap.
• Finally, running the last relevant transaction, the client has successfully synched up with the tip of the chain and can now allow a user to do interactions.

I understand that this model would bring its technical challenges too as for a contract with lots of transactions, syndicating all transactions and running them locally could become computationally intensive. But think about it in another way. By making atomic transactions the de-facto representation for state changes in the frontend also would allow much-improved syndication. Transaction's are topologically orderable and they're uniquely addressable. I can hence see large application's front ends talking via WebRTC and syndicating transactions through e.g. CRDTs. It would allow scaling the partial replication of state not through increasing a single full node's server capacity but likely scale linearly by adding more users (compare to seeders in Bittorrent).

So how do we get there?

I'm probably not the right person to ask, unfortunately. I understand well how to write applications that use Ethereum and web3. But I'm a noob when it comes to the design and architecture of the Ethereum full node clients. I've started toying around with the EVM and I thought about building a library that would allow replication of state for a simple contract. However, I believe there's a big technical challenge in finding the transactions involved in state changes or isn't there? From what I understand, another contract's internal call could as well manipulate my contract's state. But full node wouldn't allow me to query for these transactions. Right?

What's left to say is that I've looked at the light client protocol and the graph. But neither of these solves the above-described problems. I want something simple and I want something that is decentralized from an architectural viewpoint.

Any leads? What would you recommend me looking at?

[github-actions[bot]]

Since this is your first issue, we kindly remind you to check out EIP-1 for guidance.

[MicahZoltu]

You can achieve this by using events in your contract that record all contract calls. However, projects that have tried this tend to run into the problem of the cost of recreating state increases with time since you need to walk through more and more blocks and transactions as time and usage progress forward.

[TimDaub]

You can achieve this by using events in your contract that record all contract calls.

I'd consider this more of a work-around today for a project where I have control over a contract. A colleague of mine and I have worked on ERC721s in the past, where querying through events became so slow that nodes gave up while executing the RPC call [1]. But in any case, using events is not a viable option for any contract I, as a developer, am not in control of. E.g., with another project, I'd like to give users an overview of all balancer pools in a particular ecosystem. However, parts of these contracts' states don't emit events, and where I have no guarantee over them emitting events in the future either.

Finally, if possible I would like to NOT rely on heavy backends as they reduce the autonomy of the system I'm building. Ideally, all logic is handled in the frontend (e.g. in a user's browser).

However, projects that have tried this tend to run into the cost of recreating state increases with time since you need to walk through more and more blocks and transactions as time and usage progress forward.

I'm aware of that problem. However, I think it's one that front-end developers can address themselves once a future Ethereum architecture has been proposed that addresses their needs. Optimistic updates, snapshotting, and other approaches come to mind readily. As I've said before too, the web is starting to evolve towards peer 2 peer too. WebRTC is here. WebTorrents exist. There are decentralized databases like OrbitDB etc. [3]. Right now, forcing heavy reliance on the RPC endpoint limits a front-end developer's exploration radius significantly.

I've originally posted here out of pragmatism towards real problems as a front-end developer. I'd love to drive my ideas here further if there's interest. What do you propose as a first step towards a change @MicahZoltu? I could imagine gathering information from front-end developers in the field and showing it here.

References

• 1: https://github.com/vrde/erc721-benchmark
• 2: https://rugpullindex.com/
• 3: https://rugpullindex.com/blog#ScalingEthereumOneTxAtATime

[MicahZoltu]

Ideally most dapps move to layer 2, where they have a lot more freedom to build things how they like. 😄

[TimDaub]

dapps move to layer 2

I'm attempting to build on layer 2 too. It's just that instead of relying on general-purpose technology (which has failed me in the past, e.g. Plasma), I'm now trying to build an application-specific layer 2 solution designed around my app's use case. Hence the prompt for easier data sharing between full nodes and clients.

In any case, though, having worked on layer 2 myself with https://leapdao.org (minimum viable plasma), getting useful data from a full node wasn't exactly easy there either. So also layer 2 solutions could benefit from making L1 on-chain data more accessible.

[github-actions[bot]]

There has been no activity on this issue for two months. It will be closed in a week if no further activity occurs. If you would like to move this EIP forward, please respond to any outstanding feedback or add a comment indicating that you have addressed all required feedback and are ready for a review.

[github-actions[bot]]

This issue was closed due to inactivity. If you are still pursuing it, feel free to reopen it and respond to any feedback or request a review in a comment.