RFP Application: ZFile - Paying for Filecoin storage in ZEC

[MeerKatDev]

RFP Proposal: ZFile - Paying for Filecoin storage in ZEC

Name of Project: ZFile - A PoC for Paying for Filecoin storage in ZEC

Link to RFP: FF-ECC grants

RFP Category: core-dev, technical-design

Proposer: @eigerco (for GitHub reference: @MeerKatDev)

Do you agree to open source all work you do on behalf of this RFP and dual-license under MIT and APACHE2 licenses?: Yes

Introduction

We are directly responding to this Call For Proposals, particularly the Filecoin storage payments in ZEC, which will enable acquiring Filecoin storage with the level of privacy provided by Zcash shielded transfers.

GOAL: ENABLE FILECOIN USERS TO PAY FOR STORAGE PRIVATELY

The global demand for data storage continues to increase. Still, most of it is stored on centralized servers controlled by a limited number of providers, presenting security and reliability issues. Decentralized storage solutions, such as Filecoin, offer a more secure alternative by distributing data across a network of computers, reducing the risk of failure.

While Filecoin provides a decentralized storage solution, it does not offer transaction privacy. On the other hand, Zcash offers private transactions. We propose combining these two technologies' main strengths and enabling any Zcash user to pay for Filecoin storage privately.

In a reasonable timeframe, we aim to research the best ways to reach our goal and provide a reference implementation to enable even non-developer Filecoin users to provide this service on the blockchain.

It is important to emphasise that every aspect of this proposal can be adjusted and discussed.

Note that during this proposal the words Actor and smart contract are interchangeable since Actors are the Filecoin equivalent of smart contracts in the Ethereum Virtual Machine. The wording “smart contract” was used for its familiarity for any readers who understand the concept but may have not heard of Filecoin Actors.

Background

• Filecoin is one of the most ambitious blockchain projects currently operating, aiming to create a fully decentralized storage network. However, there is no privacy in any part of the blockchain. The idea to introduce the first taste of privacy to Filecoin is a great one, and it will start with paying for storage from Zcash, the pioneer in private blockchains.
• The proposal below is based on the current Zcash design. But as ZCash evolves, we can provide new alternative designs for our proposed system. The design below is modular enough to adapt future protocol changes.
• Building completely trustless systems is challenging in a decentralized world, so one must design around the known constraints and make all tradeoffs visible to stakeholders.

Components

1. User

The User is an individual or entity who wants to pay for Filecoin storage privately.

2. Escrow

The Escrow is the intermediary taking care of payments and storage, which is composed of two parts:

• An off-chain component:
  • It receives ZEC from the User
  • It pays ZEC to the Provider
  • Verifies on-chain data validity
  • The source for currency exchange information (ZEC/FIL) will come from an oracle, e.g. tellor or lilypad
  • It makes sure the Users have committed the payment needed for storage, thus ensuring that Providers get paid for storing files
  • In the future, the code used by the Escrow off-chain component, which the DAO controls, will be open-sourced, and its execution will be restricted by either Multi-Party Computation or a quorum of partial keys
• A smart contract on the Filecoin blockchain:
  • It’s to be controlled by a DAO using Multi-Party Computation or a quorum of keys.
    • The DAO could be composed of members from the Zcash and Filecoin communities
  • The Escrow smart contract acts as a middleman between the user and the Provider
  • There can be multiple providers of this service, thus creating an open market for users to choose from
  • It ensures the Provider has uploaded the file and completed its task before releasing the funds of the user, thus ensuring dishonest Providers do not run with the Users payment

The user has confidence in the Escrow off-chain component because it is governed by the DAO and in the smart contract because it is verifiable on chain. The smart contract authorizes the Providers Smart Contract by validating the on-chain contract that the Provider interacts with. The Providers are not assumed to be reliable.

3. The Provider

The Provider is the intermediary that is composed of two parts:

• An off-chain component that:
  • Stores the file in IPFS
  • Communicates with the Escrow and Provider smart contracts
  • Handle storage pricing parameters
• A smart contract on the Filecoin Blockchain
  • Interacts with the Escrow smart contract
  • Receives instructions from the Provider off-chain side

💡 **Below are examples of preliminary ideas on how these things might work. Note that the design will undergo a research phase where all the details below might change**

Provider Setup

To become a Provider and start earning on private storage requests, one must:

1. Put in place the necessary infrastructure which uploads the files on chain and handles the payment.

2. Register the Provider with the Escrow smart contract and gain access to a deployed smart contract:

   sequenceDiagram
   participant Provider off-chain component
   participant Escrow Smart Contract
   participant Provider Smart Contract
   
   Provider off-chain component->>Escrow Smart Contract: Requests to register a new provider
   activate Escrow Smart Contract
       note left of Escrow Smart Contract: The payload contains <br/>Providers Filecoin address
       Escrow Smart Contract->>Provider Smart Contract: deploy & init
       activate Provider Smart Contract
       Provider Smart Contract-->>Escrow Smart Contract: Address
       deactivate Provider Smart Contract
       note over Escrow Smart Contract: Register the deployed<br/> contract as verified in smart contract storage
       Escrow Smart Contract-->>Provider off-chain component: Address
   deactivate Escrow Smart Contract

3. Control a Zcash address.
4. Providers smart contract must always possess enough FIL to manage the load of files, which the Provider promises to handle.

Features of Provider Smart Contract

1. Automation: With a smart contract, most procedures are automated, streamlining the overall process and reducing the potential for manual errors.
2. Security: Acts as an additional layer of protection from malicious Provider activity.
3. Extensibility: Although the initial set of approved contracts will be developed and available, having multiple implementations of smart contracts allows contributions and various options. This enables anyone to propose a new "candidate", fostering a more collaborative and innovative environment.
4. Governance: Once a proposed contract is reviewed by the responsible party (e.g., the future DAO), it can become an Approved Contract. This process ensures that updates or new smart contracts align with the community's best interests and adhere to established security and efficiency standards.

In summary, Providers smart contract enhances the system's efficiency, security, extensibility, and governance, benefiting all parties involved.

A notable aspect of this proposal, which positively impacts both the Filecoin and Zcash ecosystems, is the opportunity for anyone to serve as a Provider of this service. Individuals can independently establish the necessary infrastructure and charge a fee for their services. The Escrow is designed to seamlessly integrate with such Providers, fostering a collaborative and inclusive environment.

End to end interaction

sequenceDiagram
    autonumber

    participant User
    participant Escrow off-chain component
    participant Provider off-chain component
    participant Escrow Smart Contract
    participant Provider Smart Contract

    note over User: Assumptions: <br/> Already has a CID of the file  <br/> Already has a Provider selected <br /> And storage params and prices

    User->>Escrow off-chain component: Order request <br/>+ File + ZEC transfer
    activate Escrow off-chain component
        Escrow off-chain component->>Escrow Smart Contract: Create an order for Provider to claim
        activate Escrow Smart Contract

        alt Provider succeeds
            Escrow off-chain component->>Provider off-chain component: Forward order request
            activate Provider off-chain component
                Provider off-chain component->>Provider Smart Contract: Uploads file and create a storage bid
                activate Provider Smart Contract
                    Provider Smart Contract-->>Escrow Smart Contract: Deal accepted callback
                deactivate Provider Smart Contract
                    Note over Escrow Smart Contract: Verifies storage information
                Escrow Smart Contract-->>Escrow off-chain component: Notifies about a successful deal
            deactivate Provider off-chain component
        deactivate Escrow Smart Contract
        Escrow off-chain component->>Provider off-chain component: Sends ZEC
        Escrow off-chain component-->>User: Success! <br/>Refunds leftover ZEC
        else Provider fails or any verification step fails
            Escrow off-chain component->>Escrow Smart Contract: Remove order listing
            Escrow off-chain component-->>User: Refund ZEC
        end
    deactivate Escrow off-chain component

Base assumptions: The User possesses a file (encrypted or not) that they want to store and retrieve anonymously. They have ZEC to pay for the service. The User has already precomputed the file's CID in the local client, facilitating the user experience.

1. The User aims to maintain anonymity while requesting the file upload. The solution: The User broadcasts a shielded transaction to the publicly available and transparent address owned by the Escrow off-chain component so that the memo can be read, the specifics will be determined at research phase, either a de-shielding transaction or some other way to reveal the memo. The User must:
   • Send the file and order request payload to the Escrow off-chain component;
   • Send a shielded Zcash transaction containing a memo field that associates the transaction with the file upload order.
   • The cost in ZEC is determined using price oracles, which factors in Provider fees, ZEC/FIL rates, transaction cost estimates, and an additional margin to account for network fluctuations. The User interface provides an estimate before the User takes action. The client software facilitates this User interaction.
2. Escrow off-chain component sends a transaction to the Escrow smart contract containing information about the new anonymous user order, its associated reward, and the designated Provider.
3. The Escrow off-chain component forwards the order request to the Providers off-chain component for further processing.
4. Depending on the design choice we go forward with, either
   1. The Provider off-chain component uploads the file to IPFS, retrieves the file's CID (confirming it matches the one in the Escrow smart contract order request), and funds their Provider smart contract with FIL if necessary. Subsequently, a transaction on the Filecoin blockchain creates the actual storage bid.
   2. Or the Escrow off-chain component uploads the file, but only the CID is passed along to the Provider to bid on the Filecoin storage.
5. If the storage bid is successful, the Provider smart contract communicates with the Escrow smart contract, informing it that the file has been stored. The Escrow smart contract validates that the Filecoin bid has been accepted with the appropriate parameters.
6. The Escrow smart contract notifies the Escrow off-chain component of the successful storage bid (e.g., by emitting an event).
7. The Escrow off-chain component pays the agreed-upon ZEC amount to the specified Provider's address.
8. The Escrow off-chain component may return any excess ZEC amount to the User (e.g., overcharge for covering all the fees).
9. In the event of a Provider's failure, the order will be removed from the Escrow smart contract, and the leftover ZEC will be returned to the User. A failure can happen if the Provider exceeds time bounds, creates an incorrect Filecoin storage bid, or other verification parts fail.

By following these steps, the system ensures anonymity for the User while securely storing and retrieving files on the Filecoin network using Zcash payments.

Other consideration

Fees

All fees will be calculated before the User initiates their first transaction. This will be implemented by the Escrow off-chain component, which communicates with the chosen Provider, considers blockchain transaction expenses, and consults ZEC/FIL oracle prices. The User will be responsible for covering the fees and transaction costs for all operations, which include:

• 3 Zcash transactions: from the User to the Escrow account, Escrow to the Provider, and Escrow back to the User.
• Provider fees may be set at the Provider's discretion (potentially even at zero).
• The ZEC equivalent of FIL used for storage, as determined by the most suitable Oracle provided by the Escrow smart contract at the time of storage.
• Include an extra x% margin overcharge to cover network transaction cost fluctuations.

Future DAO Organization

The FVM enables the creation of DAOs. After developing a production version, the ownership of the Escrow smart contract should be transferred to an on-chain DAO. It should be composed of members of the Filecoin community and the Zcash community.

The deployment, execution, and operation of the Escrow service and the DAO can be democratized so that it becomes straightforward for anyone to run this service and for an economy of competing escrow services to spring up. This allows users to select their preferred service Provider based on reputation or previous experiences, promoting a more user-centric and diverse market.

DAOs may also run with a service fee thus incentivizing the work done.

Development Roadmap

The development roadmap includes a research phase where we do an initial viability study and direct eventual questions to the Filecoin community.

We will also evaluate different security aspects of the system: possible attacks from any of the actors, how to protect the non-compromised actors, and how faulty blockchain nodes may impact the service.

The following months will be spent developing the product and deploying an exemplary infrastructure on a testnet.

Milestone 1 - Research and Preparation

Estimated duration: 2 months FTE: 4 Costs: $ 211 200

Goals:

• Research the technologies and Rust/Solidity libraries which could be used when developing the product.
  • Find their advantages and drawbacks.
  • Prepare charts and benchmarking results for objective comparison.
  • Develop a few smaller-scale demos to understand the possibilities of different technologies.
• Determine the project's full scope and better evaluate the viability of each component.
• Evaluate the conditions we used to declare that a Provider is misbehaving or malicious (e.g., time-bound checks, file checksum verification).
• Research the possibilities for returning user funds from the Escrow if the Provider misbehaves.
• Research approaches on how the Escrow could retrieve the latest oracle prices (e.g. using tellor).
• Research methods for robust state monitoring for Zcash and Filecoin blockchains to evaluate successful transactions.
• Research and document a detailed procedure on how to bootstrap the project - infrastructure, Provider registration, contract deployment, etc.

Requirements for Evaluation:

• A well-formatted research result document should be used for aggregating and displaying results.
• All research findings should contain detailed explanations of the findings and the steps taken to reach them.
• Charts and data tables should be used to present research data (both inputs and outputs).

Evaluation Plan:

• Internal feedback from other developers with relevant experience.
• External feedback from the Filecoin/Zcash community.

Milestone 2 - Developing Separate Components

Estimated duration: 2 months FTE: 4 Costs: $ 211 200

Note: sub-milestones are meant to be reached while working in parallel, i.e., two teams will work simultaneously.

Milestone 2.1 - Developing a Provider

Goals:

• Implement the core features needed to run the Provider process:
  • Handling of new files uploaded by the User.
  • Sending of Bid Order Requests to Provider smart contract.
  • Handling of storage confirmation events by:
  • Listening for confirmation events from Provider smart contract. Efficiently polling the data from Provider smart contract.
  • Requesting and handling ZEC payments.
• Write proper unit tests for all core features and have high code coverage.
• Have solid code conventions in place.
• Prepare CI/CD jobs for testing, linting, and building.
• Prepare a Dockerfile for cross-platform usage.

Requirements for Evaluation:

• Tests should be passing and have reasonably high code coverage.
• It should be possible to build an executable without errors/warnings and run it in a Docker container with mocked services.
• It should be possible to build an executable without errors/warnings and run it in a Linux-based environment with mocked services.
• CI/CD should be able to produce expected results (e.g., run tests and show the results in the code repository).

Evaluation Plan: Feedback from internal/external developers with relevant experience.

Milestone 2.2 - Developing the Escrow and Smart Contracts

Goals:

• Implement the core features needed to run the Escrow:
  • Handling of ZEC transactions.
  • Sending storage confirmation events.
  • Validation of Provider smart contracts version and other metadata.
  • Validation of the uploaded file’s checksum and Order Bid’s spacetime parameters.
  • ZEC payment sending.
• Implement the core features needed to run the Provider smart contract:
  • Handling of Bid Order Requests.
  • Bid Order sending to the Filecoin blockchain.
  • Sending storage confirmation events.
  • Communication with the Escrow smart contract.
• Write proper unit tests for all core features and have high code coverage.
• Have solid code conventions in place.
• Prepare CI/CD jobs for testing, listing, and building.
• Prepare a Dockerfile for cross-platform usage.

Requirements for Evaluation:

• Tests should be passing and have reasonably high code coverage.
• It should be possible to run the components in a test environment.
• CI/CD should be able to produce expected results (e.g., run tests and show the results in the code repository).

Evaluation Plan: Feedback from internal/external developers with relevant experience.

Milestone 3 - Connecting The Components

Estimated duration: 2 months FTE: 4 Costs: $ 211 200

Goals:

• Ensure that the Provider and the Escrow can be properly integrated with one another.
• Fix areas where the system had deviated from the desired path.

Requirements for Evaluation: During manual testing, everything should work as expected (outlined in the research stage).

Evaluation Plan: Feedback from internal/external developers with relevant experience.

Documentation, Education, and Community

There will be documentation to understand the various parts of the architecture and a FAQ updated after each development cycle.

We will keep the community updated via Slack and Discord on the progress and, with measure, respond to eventual questions. We regard the community as essential to the best possible outcome for the present endeavour.

Other Deliverables

We will document our research progress and release it to the community to understand how the work comes into being and the reasoning behind architectural decisions.

Milestone Summary

Milestone	Milestone Description	Funding	Estimated Timeframe
Milestone #1	Research and Preparation	$ 211 200	2 months
Milestone #2	Developing Separate Components	$ 211 200	2 months
Milestone #3	Connecting The Components	$ 211 200	2 months

Total Budget Requested

The rate for a Senior developer at Eiger is 150 $/hr per developer.

Total estimated duration: 6 months (24 weeks)

The total rate for 24 weeks with 5 working days for 8 working hours for four developers:

633 600 $

Since this is a joint grant proposal between the Filecoin Foundation and the Electric Coin Co., the total cost is expected to be split between ECC and FF.

Maintenance and Upgrade Plans

Once the work under the scope of this grant is completed, we would like to discuss further maintenance or expansion through an additional grant application. For example, we could convert the smart contracts written in Solidity for the FEVM to WASM for the FVM once it is released on the testnet.

Another upgrade could be research and implementation for a network with multiple Escrows, with the model mentioned under "Future DAO Organization."

Moreover, we would like this grant application to be considered as a beginning of a journey, where the continuation would be the next-in-line functionality mentioned on the website for FF-ECC grant proposals, "Media and file support in Zcash encrypted memos and NFTs," which would use the technology developed in the present grant proposal.

Team

Web3 promises to upgrade the very foundations of our society – from money, finance, and governance to media, gaming, and science. To deliver on that promise, decentralized technologies are to be integrated into the everyday experiences of billions of people. For engineering, this is a mountain of a challenge.

Eiger was founded to develop infrastructure for web3 mass adoption. We help technology companies improve and integrate the core technologies of web3 to meet the climbing demands for scaling and performance.

We currently employ 25+ senior web3 engineers across the globe and work with some of the most ambitious organizations in the industry, including Forte, Aleo, and XRP Labs, to name a few.

Eiger is part of Equilibrium Group, a blockchain powerhouse founded in 2018, composed of three entities:

Eiger – high-value engineering services Equilibrium Labs – applied research, proprietary products, investments in early-stage ventures Membrane Finance – infrastructure and services for EUROe, a euro-backed stablecoin

Contact Info

• luca@eiger.co (Main author of the protocol)
• daren@eiger.co (Founder & CEO of Eiger)

Team Members

The Proof of Concept will be written using Rust and Solidity. We can provide Senior developers in both technologies.

Development team

• Eloy López Peñamaría (Github, LinkedIn) is a Cloud Software Engineer with extensive experience in Rust and Go, focusing on observability and resilience. He likes to collaborate in blockchain-related projects and distributed high-traffic cloud environments.
• Roberts Ivanovs (Github, Linkedin) is a Rust Software Engineer at Eiger. He has extensive experience using Rust for performance-sensitive backend work, the IoT industry, web development, and Solidity/dApp development.
• Lauri Peltonen (Github, LinkedIn) is a blockchain developer at Equilibrium Labs. He has worked on EVM-based blockchains for five years and some 15 years in more traditional IT development. Nowadays, he builds mostly EVM dApps in Solidity and does Zero Knowledge research. You can spot him in quite many Ethereum Layer 2 Discords as well.
• Jani Anttonen (GitHub, LinkedIn) is a Software Engineer at Equilibrium Labs. He has over a decade of experience in web development, plus Solidity/dApp development, as long as DeFi has been a thing. An experimenter at heart, lately, he’s been involved in developing a multi-sig wallet for Starknet.

Advising team

• Luca Campobasso (GitHub, LinkedIn) is a Software Engineer at Eiger. He has extensive experience with functional languages (7 years), especially with Scala and Elixir, primarily focused on backend systems design. He conceptualized the described protocol.
• Niklas Long (GitHub, LinkedIn) profoundly cares about open source, distributed systems, and data privacy. He has been on the Aleo Core Team since late 2020, mostly overhauling, testing, and profiling their network; he is the author of Ziggurat and an active contributor to Rust IPFS.
• Vesa-Ville (Github, Twitter), the CTO at Equilibrium, has decades of experience in distributed systems, databases, and interoperability solutions. As a former co-founder of Haja Networks, he studied the atomic cross-chain composability problem and developed Ambients for executing strongly confluent distributed programs. His latest projects involve privacy-oriented L1s, ZK-proof systems, and BFT protocol development.
• Mark Henderson (GitHub, Twitter) is the VP of Engineering at Equilibrium. He has led the team, starting with the original Rust IPFS grant in late 2019, through engagements with many of the largest names in Web3. He is now circling back to finish the critical work the team started with the original Ziggurat proposal. Core contributor to OrbitDB, Rust IPFS, and Ziggurat.

Team Website

Main implementer: Eiger Tech & Knowledge support: Equilibrium

Relevant Experience and repositories

As mentioned in the Teams section, Eiger already has extensive experience developing large infrastructural projects. Some chosen examples:

• The SnarkOS for Aleo, the privacy-focused smart contract L1 platform going to production this summer.
• OrbitDB is a distributed, p2p, serverless database.
• The rust implementation of IPFS, now archived.
• The main implementation of Interledger in Rust, which we developed and maintained.
• Ziggurat. Network protocol testing framework for Zcash, XRP, and Algorand. Notably, critical network vulnerabilities were found and reported to the core teams.

Moreover, Eiger is already a recipient of a Zcash Foundation grant, the UniFFI library, whose implementation is ongoing at the time of writing.

[MeerKatDev]

Hello, we are submitting this as a joint grant for Zcash and Filecoin. @mdr0id @whyrusleeping

[ErinOCon]

Hi @MeerKatDev, Thank you for your proposal and for your patience with our review. Unfortunately, we will not be moving forward with a grant at this time. While this specific RFP has closed, you may have an interest in reviewing open grant opportunities for your future work.

Thank you again for your interest in our grants program!