IWillPersist, Incrementally Encrypted Databases over IPFS

[fare]

Open Grant Proposal: IWillPersist, Incrementally Encrypted Databases over IPFS

• Name of Project: IWillPersist
• Proposal Category: app-dev
• Proposer: fare
• Do you agree to open source all work you do on behalf of this RFP and dual-license under MIT and APACHE2 licenses?: Yes

Project Description

We propose to build atop Filecoin ecosystem our Incrementally Encrypted Database technology, IWillPersist.

IWillPersist ensures that a user's private data will persist even when their browser session is interrupted, their browser is updated, their computer loses power mid-operation, their disk is corrupted, their laptop is stolen, their datacenter is closed by a government agency or destroyed by an invading army. Such persistence is very important for all the blockchain data they use, not just their many master keys, but active subkeys derived from their master keys, contact databases, lists of hidden accounts, etc. Persistence is even more important when using State Channels, and more generally for any multi-step interaction with a smart contract, wherein failing to complete your next step can result in the loss of assets.

IWillPersist allows application programmers to just assume that their data will persist, while users can choose which set of data storage providers will hold backups of their data. Database rows are incrementally encrypted, such that a 200KB transaction doesn't require reencrypting an entire 1GB database and sending it over the network. Instead, each of many rows under 4KB are encrypted and stored over IPFS with a small fee paid to lease space on each guaranteed backup provider, while space lease on deleted rows is simply not renewed.

Only the user with his master password can view or modify the database. Providers cannot read the data, nor tamper with it, only at worst drop the data or try to serve an old version. As long as the user or at least one of his chosen and paid backup provider keeps and serves the latest version of the data, the user data is safe, whereas the cheating providers can be detected and hopefully punished.

The merkle root of the metadata describing who are the current providers for a given user can itself optionally be stored on a DHT, or on a blockchain smart contract for extra security. Which DHT or blockchain is used can itself be optionally stored in a few bits of the master password.

The first version of IWillPersist will only offer a NoSQL interface to a key value store plus a content-addressed store. A latter version of IWillPersist will build a SQL database on top of the above, with tables, indexes and SQL queries, by porting some existing SQL engine (probably sqlite).

Value

The technical capabilities of IWillPersist make it ideally fit to store data for Decentralized Applications (DApps). The threat model corresponds to events that have actually happened to DApp users and are likely to happen over and over again.

By using the Filecoin infrastructure, users can have an integrated solution to store the data they use in their DApps and the payment for backups of this data.

Filecoin will benefit both from increased storage demand and from improved DApp capabilities.

Deliverables

For this proposal, we will build a simple prototype of persistent store with replication over IPFS.

The first deliverable will be a prototype that will include the following:

• A read-write key-value store
• A write-only content-addressed data store
• Local storage in a local database
• Synchronous remote backup over IPFS
• User-configuration for which paid commercial IPFS service to use
• Automatic backup restoration when the local database fails.
• A working example using the prototype
• A video demonstration of the prototype working
• Documentation of steps to reproduce the demonstration
• All of it written in Gerbil Scheme

Specifically not included in the first deliverable, though intended for later implementation:

• Support for JS, C, Rust, Go, Java and other common development languages
• A stable API for DApp clients
• Extensive Documentation on how to use the API
• Support for multiple remote backups
• Metadata storage to automatically identify the backup storage providers
• Infrastructure to detect and denounce bad providers
• Data schemas, tables or indexes on top of the encrypted row stores
• SQL queries for the data

We estimate the initial prototype to be doable in 3 months by 2 developers plus management support, at a cost of $60,000.

Development Roadmap

Our development will include the following phases.

Phase 1: Specification and Evaluation

We will write a design document as a Markdown file, explaining what will be the initial persistence API, what will be the initial local persistence backend, and who will be the initial IPFS remote backup provider. This phase will include writing code to experiment with existing APIs and providers, but without any attempt at building a usable API. The document may not perfectly reflect the final product, but will be updated as our expectations face reality.

Phase 2: Initial Working Prototype Implementation

In Phase 2, one developer will focus on the encrypted data model and user-facing API, while the other developer will focus on remote storage using IPFS. Together they will produce a working prototype and an example.

Phase 3: Bug Fixes, Demonstration and Documentation

In Phase 3, the developers will fix the bugs found during Phase 2, then produce the video demonstration and minimal documentation.

Total Budget Requested

We estimate this project as occupying two full time developers plus technical and managerial support, for three months, at a cost of $60,000.00.

If Filecoin likes the prototype we show at the end of this period, it could then fund the further development of IWillPersist from a prototype into a full-fledged robust product.

Team

Contact Info

alex@mukn.io

Team Members

• François-René Rideau, President & Chief Scientist
• Alexander Smart, Chief Executive Officer
• Peter Hubshman, Chief Financial Officer
• Zoe Braiterman, Chief Information Officer
• Gauthier Lamothe, Chief Communication Officer
• Alex Hochberger, Chief Product Officer & Chief Technology Officer
• Marcin Grzybowski, Vice President of Formal Methods
• Donald Fisk, Senior Scientist

Team Member LinkedIn Profiles

• https://linkedin.com/in/fahree
• https://linkedin.com/in/alexandersmart
• https://www.linkedin.com/in/peterhubshman/
• https://www.linkedin.com/in/zoebraiterman/
• https://linkedin.com/in/gauthier-lamothe
• https://www.linkedin.com/in/alexhochberger/

Team Website

https://mukn.io

Relevant Experience

Relevant previous work by our contributors are listed in the Wiki for our Glow language: https://github.com/Glow-lang/glow/wiki/Bibliography-Glow

We also have successfully worked with Filecoin in a past grant: Glow on Lurk

Team code repositories

Our Glow programming language: https://github.com/Glow-Lang/glow

The Gerbil-Persist layer of Glow in that already implements part of IWillPersist: https://github.com/fare/gerbil-persist

Additional Information

MuKn is already a partner of Filecoin, as a member of the Lurk project: we have implemented a prototype backend of our smart contract language Glow on top of Lurk, and have been working on the formalization of Lurk, for which we have submitted another proposal. https://github.com/filecoin-project/devgrants/issues/405

[fare]

@BlocksOnAChain

[BlocksOnAChain]

This is not under my domain since It's on the IPFS side. @realChainLife and @ErinOCon can help provide the best person to help you with this proposal @fare

[ErinOCon]

Hi @fare, thank you for your patience with our review. Unfortunately, we will not be proceeding with a grant at this time. Wishing you all the best as you continue to build!