Injecting Ethereum into Filecoin

[levlite]

Open Grant Proposal: Vulcanize - Injecting Ethereum into Filecoin

Project Name: Injecting Ethereum into Filecoin

Proposal Category: Integrations

Entity Name: Vulcanize, Inc.

Proposer: @levite

Filecoin ecosystem affiliations: Protocol Labs

Technical Sponsors: @anjor and @ribasushi

Do you agree to open source all work you do on behalf of this RFP under the MIT/Apache-2 dual-license?: Yes

Project Summary

The goal of the project is to inject the complete Ethereum state into Filecoin. This entails a multi-step process, with incremental steps towards the ultimate goal of supporting the below capabilities:

• Analysis of historical block information
• Rebuilding a full archive node
• Accessing expired danksharding blobs in a post EIP-4844 world
• Rebuilding a full node from bound historical data execution clients in a post-EIP 4444 world

With this proposal completed, we expect these new capabilities to benefit both the Filecoin and Ethereum communities and demonstrate how Filecoin can serve as a Data Availability layer to all blockchain ecosystems.

Impact

Benefits for Filecoin and Ethereum as well as key preparation for a post 4444 and 4844 world.

Outcomes

Milestone 1: Load execution layer Ethereum DAG into Filecoin Milestone 2: Load beacon chain Ethereum DAG into Filecoin

Not in the scope but subsequently possible and recommended based on this grant:

Milestone 3: Optimize storage format for analysis of historical block information and rebuilding Ethereum archives Milestone 4: Indexing, caching, and retrieval services for these use-cases Milestone 5: Provide support for future use cases (EIP-4844 and EIP-4444)

Development Roadmap

Milestone 1: Load execution layer Ethereum DAG into Filecoin

1. Store entire Ethereum DAG in Filecoin
   1. Code for storing Ethereum IPLD blocks on Filecoin
      1. Segments and transfers data from local Ethereum IPLD blockstore into Filecoin
         1. https://github.com/cerc-io/ipld-eth-db
      2. Minimize number of sectors used
      3. Minimize number of sectors that must be accessed
      4. Each sector should store the transactions, receipts, headers, and state diffs (state, storage, code) for a block range (the max block range that can fit in a sector- possibly 1 block). This optimizes retrieval based on block height when the goal is to retrieve the full set of chain and state data.
      5. Alternatively, it may make sense to store the entire set of only a single Ethereum object type for a maximum block range in a sector. This optimizes retrieval based on block height when the goal is to retrieve only a single type of chain or state data.
      6. We also store complete state snapshots separately for every nth block, or embedded into every mth sector used to store the above objects
         1. E.g. every payload containing the 32768th (CHT interval) block will include the entire state and storage tries under the root state hash CID rather than the differential set.
            1. Block 0 will include the full genesis state.
      7. By storing the entire Ethereum data set in the above formats, the root CIDs in each payload can be derived from the root hashes found natively in Ethereum.
         1. A basic retrieval market index that map Payload root CIDs will therefore maintain a
   2. A simple index is maintained around the payload and piece CIDs for this data, mapping block range boundaries and some additional metadata to all the pieces and the payload CIDs they contain.
      1. Index can be extended to support more specific needs as required, as in Milestone 3
   3. Note: local blockstore is only execution layer IPLD objects at this time
   4. Full stack integration & retrievability tests
      1. Test SPs deliver the content (by CID)
      2. Test content is indexed appropriately (in IPNI and similar)
      3. Test content gets delivered by gateways, etc.
2. Transaction retrieval for modified EIP-4444

Milestone 2: Load beacon chain Ethereum DAG into Filecoin

1. Finish support for beacon chain
   1. Code for extracting beacon chain into and indexing in local blockstore
      1. Mostly complete: https://github.com/cerc-io/ipld-eth-beacon-indexer
   2. Repeat milestone 1 for storing and maintaining basic indexes around beacon chain data on Filecoin
   3. IPLD codecs for beacon chain objects
      1. Continuation of https://github.com/vulcanize/ts-dag-eth and https://github.com/vulcanize/go-codec-dageth
      2. Optional- codecs are not needed for loading the data into Filecoin
2. Blob retrieval for EIP-4844

Total Budget Requested

USD $98,400 for Milestones 1 & 2

Milestone	Description	Deliverables	Completion Date	Funding
1	Load execution layer Ethereum DAG into Filecoin		Ethereum index & full stack integration for Filecoin	$ 49,200
2	Support for Beacon Chain	Index of Beacon data for Filecoin	Simple index & full stack integration	$ 49,200

Maintenance and Upgrade Plans

The third, fourth, and fifth Milestones can be better fleshed out upon completion of Milestones 1 & 2.

Milestone 3: Optimize storage format for analysis of historical block information and rebuilding Ethereum archives

(Estimates will be clarified by Milestones 1 & 2, TBD - not in scope for this dev grant proposal)

1. More optimal storage, indexing, and caching of Ethereum data for below use cases: a. Analysis of historical block information i. Build and maintain more useful indexes around Filecoin data stored in Milestones 1 and 2
   1. E.g. maintain a mapping of transaction hashes to the piece/payloadCID they were stored with; maintain mapping of all the accounts that sent or received transactions in a given piece/payload. b. Rebuild a full archive node i. Execution layer ii. Consensus layer iii. Potentially store a format of Ethereum data on Filecoin that does not represent the full Ethereum DAG as IPLD
   2. E.g. directly store levelDB archive file(s); filesystem snapshots; aggregated IPLD payloads

Milestone 4: Indexing, caching, and retrieval services for these use-cases

(Not in Scope for this dev grant proposal)

1. Retrieval service for the above use cases a. Indexing, caching, and retrieval services for each use case

Milestone 5: Provide support for future use cases (EIP-4844 and EIP-4444)

(Not in Scope for this dev grant proposal)

1. Support for future use-cases: a. Accessing expired danksharding blobs in a post EIP-4844 world i. Split blobstore from consensus layer, store mapping in retrieval market and Filecoin ii. Note: requires consensus layer data from Milestone 2
   1. Note: EIP-4844 is not active yet b. Rebuilding a full node in a post-EIP 4444 world i. Split transaction store from execution layer, store mapping in retrieval market and Filecoin ii. Using Ethereum transaction data stored and indexed in Milestones 1 and/or 3
   2. Note: EIP-4444 is not active yet

Team

Vulcanize Inc. Cerc Inc.

Team Members

Rick Dudley, President Ian Norden, Engineering VP Erik Dies, Product Lead Leah Light, Operations Manager

Team Member LinkedIn Profiles

https://www.linkedin.com/in/afdudley/ https://www.linkedin.com/in/ian-s-norden/

githuhbs:

@afdudley @i-norden @erikdies @levlite

Team Website

https://vulcanize.io/ https://cerc.io/ https://laconic.com/

Relevant Experience

About Vulcanize / Laconic:

Vulcanize, Inc. are blockchain and IPLD experts with many years of experience in Ethereum and CosmosSDK, as well as a solid understanding of blockchain and database fundamentals.

The Vulcanize team works on Laconic, a collection of software that empowers Web3 developers to access multi-chain data in a way that uniquely preserves the verifiability of the data they extract. This service is powered by their open source IPLD-based Postgres database. It has a full index of Ethereum chain data and they are in the process of adding additional chains. The Laconic stack relies on IPLD, leverages IPFS, and intends to use Filecoin for archival storage. The repositories that make up the Laconic Stack are listed in the relevant repos below.

The Laconic product suite also includes Watchers, which auto-generate code from Solidity source, capable of reading and transforming activity from smart contracts into IPLD blocks stored and indexed in a relational database. Using the Laconic techniques, Vulcanize was able to accelerate a number of services, in particular, serving The Graph subgraphs and RPC endpoint queries.

The Laconic work originated with VulcanizeDB, a technology conceived while Vulcanize was working with MakerDAO in 2017. Since then, Vulcanize has evolved into the Laconic stack, but the original repo is still available at MakerDao.

During the process of developing the Laconic stack, Vulcanize developed the stack-orchestrator, which simplifies testing and deployment. This tool can be used by other projects to simplify the testing and deployment of the complex client and add-on services often found in the blockchain space. While testing code related to its current Protocol Labs grant, Vulcanize added Lotus to the stack-orchestrator, making it far easier for engineers to spin up local testnets for testing.

Vulcanize also developed a tool, chain-chunker, which serves to manipulate blockchain client databases. It manages clusters used to process the historical data from large chains such as Ethereum; Vulcanize intends to add Lotus support to chain-chunker in the future.

Since 2016, Vulcanize has been awarded a plethora of prestigious grants and contracts. They were core contributors to Ethereum, IPLD / IPFS, and CosmosSDK. Vulcanize received a grant from the Ethereum Foundation for EIP-1559. Vulcanize was twice appointed by the Interchain Foundation for Core Contribution to the Cosmos SDK, where they worked on SMT and State Streaming (ADRs 38 and 40). The Interchain Foundation also awarded Vulcanize a grant to work on Tendermint-Cosmos IPFS integration and to develop a Postgres-IPFS caching layer. This work was never completed, but produced IPLD schemas for the Tendermint and Cosmos data structures, as well as a Golang implementation of IPLD codecs for these objects, and a proposed scheme for providing generic IPLD support for any CosmosSDK chain. Vulcanize President Rick Dudley also served on the TAC (Technical Advisory Committee) for Interhain Foundation for several terms.

Vulcanize also received a grant from MolochDAO to build an IPLD-based Ethereum block explorer.

Currently, they are receiving funding from MetaMask / ConsenSys for their work on the MobyMask Ethereum light client and have also made major contributions to js-libp2p in the course of this project. Additionally, they are building a Nitro-based payment channel L3 for the MobyMask project.

Vulcanize has received prior large grants from Protocol Labs for their “Ethereum as IPLD” work. This work produced IPLD schemas for the Ethereum data structures as well as Typescript and Golang implementations of IPLD codecs for these objects.

Vulcanize also received a grant in 2022 to research storing blockchain state on IPFS/Filecoin for ten popular blockchains, funded by Radius.space. The final report for this project is linked here.

Currently they have a grant from Protocol Labs for a generic indexing and caching solution for Filecoin chain and state data, detailed here. This project involves work across multiple Protocol Labs repositories, including lotus, go-datastore, boxo, go-ipld-cbor, go-amt-ipld, go-hamt-ipld, and others.

References Filecoin Extractor Project Plan for a generic indexing and caching solution for Filecoin chain state data (current grant)

Cosmos IPLD Project Plan

• While Vulcanize is committed to open source work, some of the relevant repositories are currently privately hosted on Gitea.

Team code repositories

Relevant repos *

IPLD schemas and codecs: https://github.com/vulcanize/ts-dag-eth https://github.com/vulcanize/go-codec-dageth https://github.com/vulcanize/go-codec-dagcosmos https://github.com/ipld/ipld/tree/master/specs/codecs/dag-eth https://github.com/ipld/ipld/tree/master/specs/codecs/dag-cosmos

Laconic Stack: https://github.com/cerc-io/ipld-eth-db https://github.com/cerc-io/ipld-eth-server https://github.com/cerc-io/go-ethereum/tree/v1.11.6-statediff-v5/statediff https://github.com/cerc-io/eth-statediff-service https://github.com/cerc-io/ipld-eth-state-snapshot

Watcher related: https://github.com/vulcanize/watcher-ts https://github.com/cerc-io/gelato-watcher-ts https://github.com/cerc-io/mobymask-v2-watcher-ts https://github.com/cerc-io/ts-nitro

Stack orchestrator: https://github.com/cerc-io/stack-orchestrator (including Lotus support)

Filecoin indexing: https://github.com/vulcanize/filecoin-indexing https://github.com/vulcanize/lotus-index-attestation

Additional Information

Eva Shon leah@vulcanize.io

### Tasks

[ErinOCon]

Hi @levlite, thank you for your proposal and for your patience with our review. Your application is still under review, but we expect to have more information available in the coming weeks.

If we have any questions for your team, we will be sure to connect!

[ErinOCon]

Hi @levlite, thank you again for your patience with our review. Unfortunately, we will not be proceeding with a grant at this time. Wishing you all the best as you continue to build!