search

status-im / swarms

Swarm Home. New, completed and in-progress features for Status
92 stars 31 forks source link

Nimbus - An Ethereum 2.0 Sharding Client #67

Closed 0xc1c4da closed 6 years ago

0xc1c4da commented 6 years ago

Preamble

Idea: <to be assigned>
Title: <title>
Status: <Draft | In Progress | Closed >
Created: <date created on, in ISO 8601 (yyyy-mm-dd) format>
Requires (*optional): <Idea number(s)>
Replaces (*optional): <Idea number(s)>

[]{#_neozr341c9fa .anchor}Nimbus

[]{#_6iswzp4rjc4e .anchor}An Ethereum 2.0 Sharding Client\ Draft 25/12/17

OVERVIEW

Nimbus aims to be a Sharding client implementation for the Ethereum Public Blockchain. Nimbus will be designed to perform well on resource-restricted devices, focus on a production-ready implementation of Web3 and will be supported and maintained to deliver on all of Ethereum 2.0's goals.

GOALS

  1. Create an Ethereum Implementation suitable for Resource-Restricted

    Devices.

  2. Create an Implementation team for [[Ethereum

    Research]{.underline}](http://ethereumresearch.org/)'s (aka Ethereum Asia Pacific Limited) [Applied Research Objectives]{.underline}. With a focus on:

    a. Proof-of-Stake

    b. Sharding

    c. Stateless Clients

    d. LES2 (?)

  3. Close the Gap between Research Modeling and Production.

  4. Pledge to participate, implement and conform to the EIP process.

  5. Permissive Licensing.

  6. Focus on Production-ready Web 3.0 Stack (Whisper, PSS and Swarm) and

    it's continued research & development.

  7. Marketing & promotion to address community concerns on scalability

    and bolster Ethereum's dominant mindshare.

REQUIREMENTS

Nim

[Nim]{.underline} is a productive, general purpose systems programming language with a Python-like syntax that compiles to C. Using Nim allows us to rapidly implement Ethereum and allows us to take advantage of mature C tooling in both compilation of machine code and static code analysis.

With Ethereum research currently being modeled in Python, the end result should be code that is both easy for us to bring research into production, a high degree of code reasonability for researchers, as well as being performant for production.

In addition to this the core contributors and Nim community have been very supportive and enthusiastic about support for the project.

For more information please read:

[https://nim-lang.org/]{.underline}

Development on Embedded Systems

We believe that the largest successful deployment of Ethereum will reside on embedded systems, specifically mobile personal computing (Smartphones) and IoT devices.

Existing implementations of Ethereum have been focused on Desktops and Servers, while necessary for the initial success of Ethereum and being suitable for full & archival nodes, their deployment onto embedded systems has been an after-thought. Through the development of Status we have found the dominant Ethereum implementations, Geth and Parity are not suitable candidates for our target platform without profiling and optimisation (in progress).

While archival nodes with Nimbus will be supported, Nimbus will be developed as a light client first implementation with a focus on Proof-of-Stake and Sharding.

Through the deployment of Status among 40,000 alpha testers, a significant portion (23.6%) of users still run on old mobile devices. We propose a self-imposed constraint, a requirement to build and run on 2014 SoC architectures, the Cortex-A53 (Samsung Note 4 & Raspberry Pi 3) and the Apple A8 (iPhone 6).

With the 2020 scalability goal fully realised, this will ensure Ethereum runs performantly on at least 6 year old resource-restricted hardware.

Extensible, Configurable & Modular Design

The application architecture should have modular abstractions for the following:

  1. Networking Layer

  2. Sub-protocols

  3. Consensus

  4. Privacy

  5. Database

  6. Virtual Machine

And should be built against the Common Compliance Tests

[https://github.com/ethereum/tests]{.underline}

Ethereum Improvement Proposals Commitment

Nimbus is committed to open standards and maintaining consensus with other compliant Ethereum implementations. Nimbus will do its development and protocol changes via the Ethereum Improvement Proposal process.

[https://github.com/ethereum/EIPs/]{.underline}

User Experience

Access to Shards and Mainchain state should be fast & responsive, the application binary should be lightweight in terms of resources used, and the client should be dependable & robust against crashes.

Apache v2.0 Licensing

Another unsolved hurdle Status has faced is the LGPLv3 License as its Runtime Linking requirement is incompatible with mobile app distribution channels (Apple App Store).

Despite numerous requests for a static-linking exception over the past year, it has not materialized, this blocks the ability for any legally sound full Ethereum client being deployed on mobile devices. Furthermore LGPL prevents adoption of Ethereum on closed hardware platforms such as XBox.

We propose to license Nimbus under Apache 2.0, a permissive license with patent protection that will further the reach of the Ethereum platform and ensure highest degree of adoption with governments and enterprise.

Weekly Development Reports, Technical Writing & Promotion

In addition to the implementation, Nimbus will have a weekly process on reporting development updates.

A technical writer that will translate stable research discussion into articles more easily understood by the community as well as document the implementation.

In addition to this, we will also promote Ethereum as the leader of scalable Public Blockchains within the crypto-community.

Bounty-based Development

Tasks that can be self-contained and well-defined will be put up as Github issues and bounties will be attached to them to entice the community to further accelerate development.

MILESTONES

Timelines are approximate, will be affected by research and implementation considerations and will be revised during the team's production of a detailed implementation timeline.

Team Formation & Detailed Project Implementation Timeline

January - February 2018

Initial Team Formation, defining project scope, architecture and implementation timeline.

Milestone Goals:

  1. 5 full-time core contributors.

  2. up to 5 part-time core contributors.

  3. 1 Project Manager

  4. 1 Technical Writer

  5. Detailed implementation timeline of the project as deliverable.

Aim of 10 full-time core contributors by 2019.

Current Team, Start Date:

Zahary Karadjov - 2nd Jan

Alexander Ivanov - 2nd Jan

Mamy Ratsimbazafy - 1st Feb

Alpha & Reference Implementation Knowledge Transfer

January - \~November 2018

Due to code similarity with Python and [Py-EVM]{.underline} being essentially a refactor of [PyEthereum]{.underline}. We will initially port Py-EVM in Nim which will serve as developing code familiarity and inform initial application architecture. Existing implementations ([go-ethereum]{.underline}, [pyethereum]{.underline}, [py-evm]{.underline} and [Parity]{.underline}) will be referenced and understood before implementation in Nim.

Milestone Goals:

  1. Alpha Ethereum Sharding client implementation as deliverable.

  2. Team familiar with all Ethereum implementation Codebases

  3. Team understanding main themes from

    [ethresear.ch]{.underline} & actively participating in [EIPs]{.underline}.

Beta & Security Auditing

\~November 2018 - \~March 2019

Milestone Goals:

  1. Deliverable of security audited & production-ready client of

    expected Sharding phase.

Whisper & PSS Implementation

July - October 2018

Bounties and advertised will be put up as soon as p2p layer has been implemented, otherwise will be started in July by core team. If IPFS is considered, IPFS PubSub may be used instead.

Swarm Implementation

October 2018 - \~March 2019

Bounties and advertised will be put up as soon as p2p layer has been implemented, otherwise will be started in October by core team. If devp2p gives way to libp2p, IPFS will be considered only on approval from Ethereum Foundation.

QUESTIONS

  1. Will devp2p be phased out in favour of libp2p?

    a. Unclear at this moment.

  2. What is the future of Swarm?

    a. Current understand is Swarm is continuing to move forward. That

    said, one possible consequence of sharding is that in the long term, there will be a large need for a Swarm-like incentivized storage system for the entire blockchain (+state and history), so it could be reasonable to try to merge the effort for that and the effort for building a general-case incentivized storage system, and just have them work off of the same protocol

  3. Is LES2 relevant in a Stateless Client paradigm?

    a. Something like LES is definitely very relevant. Remember that in

    a stateless sharded paradigm, every node is a light client on most shards most of the time, so having a working and efficient light client protocol is super-important

  4. What are the preferred communication channels?

    a. Skype and telegram; we can also set up shared skype/gitter rooms

RESOURCES

  1. [https://github.com/ethereum/sharding/blob/develop/docs/doc.md]{.underline}

  2. [https://www.youtube.com/watch?v=9RtSod8EXn4&feature=youtu.be&t=11493]{.underline}

  3. [https://ethresear.ch/t/the-stateless-client-concept/172]{.underline}

  4. [https://www.youtube.com/watch?v=hAhUfCjjkXc]{.underline}

  5. [https://github.com/ethereum/py-evm/issues?q=is%3Aopen+is%3Aissue+label%3ASharding]{.underline}

  6. [https://github.com/ethereum/py-evm/pulls?q=is%3Aopen+is%3Apr+label%3ASharding]{.underline}

  7. [https://github.com/ethereum/py-evm/tree/sharding]{.underline}

  8. [https://ethresear.ch/c/sharding]{.underline}

  9. [https://gitter.im/ethereum/research]{.underline}

  10. [https://gitter.im/ethereum/py-evm]{.underline}

IMPLEMENTATION THOUGHTS

Need to create [devp2p]{.underline} (and abstraction to potentially allow for [libp2p]{.underline}), [Node Discovery]{.underline}, [RLP encoding]{.underline}, [Modified Patricia Merkle Tree]{.underline} , [bigint's]{.underline}, keccak256 and secp256k1.

Ontop of this we need an abstraction to allow for sub protocols ([SHH]{.underline}, [PSS]{.underline}, [Swarm]{.underline}, [LES]{.underline}, Plasma(?)/State Channels)

DB: Most Eth implementations use Leveldb, Parity has a db abstraction and uses hashdb and rocksdb.

Rocksdb is an interesting choice as it solves issues leveldb comes in contact with and they have a [lite version]{.underline} for mobile usage but is C++ which is an issue only if we go for pure C.

EVM virtual machine - basic vm, [eWASM]{.underline} (Hera is also C++)

IPC/RPC abstraction, [external API methods]{.underline} that can be consumed by application bindings (react-native module, IPC, RPC http server or web sockets)

Encryption Library is a little unclear, Libgcrypt has everything we need but may be problematic LGPL licensing standpoint, we could use it now if we have abstraction for it and swap it out later for something more permissive, or roll our own (not a great idea to do own encryption will definitely need to be audited and tested if we do), open to suggestions.

Need to monitor [https://github.com/ethereum/py-evm/tree/sharding]{.underline} and connect with people working on sharding

Summary

Swarm Participants

Product Overview

Product Description

Requirements & Dependancies

Minimum Viable Product

Goal Date:

Description:

Dates

Goal Date:

Description:

Testing Days required:

Success Metrics

Copyright

Copyright and related rights waived via CC0.

oskarth commented 6 years ago

Closing this issue as part of spring cleaning. If this idea is still relevant, please submit a PR per https://github.com/status-im/ideas/#contributing. If you feel closing this issue is a mistake, feel free to re-open.