Open Grant: Decentralized Edge Compute Cloud

deMEC Cloud Computing

integration-adoption

Proposer: fruitbox12

Do you agree to open source all work you do on behalf of this grant and dual-license under MIT, APACHE2, or GPL licenses?: Yes

Project Description

Our project aims to solve the challenges of securely and reliably executing distributed workloads at scale in a decentralized manner. To address this need, we are building a decentralized Cloud Services Platform, powered by a EVM-agnostic MEC (Multi-Access Edge Computing) protocol that leverages the power of Lit Protocol's public key management contracts and the Ethereum Virtual Machine (EVM) to gate access to resources on the network.

Incorporating virtualized routing functions as WebAssembly builds into each node of the MEC system provides several benefits. First, it can provide much better performance compared to traditional validator nodes and EVM nodes, as the virtualized routing functions can be optimized for the specific hardware and software environment in which they are running. This can result in reduced latencies and improved overall performance of the network. Additionally, using virtualized routing functions as WebAssembly builds allows for more flexible and scalable deployment of these functions, as it is much easier to deploy and scale these functions in a virtualized environment compared to traditional hardware-based implementations.

However, traditional P2P networks and DHTs often suffer from a lack of native support for Network Function Virtualization (NFV) and virtualized routing functions. This can result in reduced performance and scalability of these networks. Our decentralized MEC system addresses this issue by leveraging Lit Protocol's key distribution and access control mechanisms to securely distribute and gate access to resources on the network. This allows for the deployment and execution of virtualized routing functions and other network functions in a scalable and secure manner.

As a backbone, we opted for an Append-Only Log Based Solution with Native Support for Distributed Hash Tabling to create an off-chain overlay network with it's own respective set of RPC methods and routing strategies, specifically for arbitrary compute with a Platform-As-A-Service Model/User Experience. Users are empowered to provide infrastructure with a given SLA and dynamic pricing such that they are paying for resources at actual market value with the given terms and agreements for A. Risk B. Uptime and Availability and C. Location and Distribution

Hypercore Protocol is an open framework and peer-to-peer data sharing solution that provides much higher level and web-specific implementation compared to other P2P Data Protocols such as IPFS. Hypercore Protocol is a decentralized data storage and distribution system that allows users to create and share append-only logs, known as hypercores. These hypercores can be used to store and distribute data in a secure and decentralized manner, making it well-suited for use in edge computing applications.

Hypercore Protocol uses public and private keys to authenticate users and control access to hypercores. Public keys are used to identify users and can be shared publicly, while private keys are used to sign and verify messages and are kept secret by the user. The lifecycle of these keys can be managed using signed messages, such as those used in ERC721 transfers and mints.

In terms of edge computing, Hypercore Protocol can be used to power the distribution and execution of functions and containers at the edge of a network. By storing and distributing these functions and containers via hypercores, it is possible to execute them on remote peers in a decentralized and secure manner. This can enable a wide range of edge computing use cases, including the execution of serverless functions, the deployment of containerized workloads, and the management of event-driven processing. Overall, Hypercore Protocol's key management and data distribution capabilities make it well-suited for use in decentralized edge computing applications.

We believe that our solution is well-suited to solve the challenges of decentralized computing at scale because it leverages the security and flexibility of Lit Protocol's public key management contracts and the EVM, as well as the scalability and reliability of a DHT based on hypercores. The incorporation of virtualized routing functions as WebAssembly builds further enhances the performance and scalability of the MEC system, while Lit Protocol's key distribution and access control mechanisms ensure that the network remains secure and resources are effectively managed

Lit Protocol's contracts can be used to verify the authenticity of signed messages and events, ensuring that only authorized parties are able to access resources on the network.

The protocol's support for various types of public keys, including JWTs, allows for a flexible and secure method of identifying and authenticating parties on the network.

The use of Lit Protocol's contracts to manage access to resources can help to ensure that the execution of serverless functions, OCI containers, and other workloads is authorized and in line with the desired service level agreements.

By using the protocol's public key management contracts to gate access to resources on the network, it is possible to create a secure and reliable decentralized MEC system for executing distributed workloads at scale.

The ability to split workloads into "actors" and "capabilities" and stream their execution to verifiers via signed IPC streams can further enhance the security and reliability of the system, as it allows for the integrity of the workloads to be independently verified. Including the use of browsers as light nodes for further availability redundancy can further improve the reliability and scalability of the decentralized MEC system by increasing the number of available computing resources at the network edge and enhancing overall availability.

To establish WebRTC connections for decentralized MEC systems, it is possible to use keys associated with wallet ownership and the EVM to create a secure and decentralized method of establishing connections without the need for STUN and TURN servers. This approach can also provide additional benefits such as increased security and reduced reliance on centralized infrastructure.

By leveraging node discovery and decentralized computing, it is possible to further enhance the scalability and reliability of the MEC system by allowing for the discovery and utilization of additional computing resources on the network.

Value

The purpose of this project is to enable users to execute functions and containers on remote peers in a decentralized and secure manner. This can be used to enable a wide range of edge computing use cases, including the execution of serverless functions, the deployment of containerized workloads, and the management of event-driven processing.

The value of this project lies in its ability to provide a decentralized and secure platform for executing distributed workloads at scale. By leveraging Lit Protocol's public key management contracts and the EVM, we can ensure that access to resources on the network is properly gated and controlled. Additionally, the use of virtualized routing functions as WebAssembly builds enhances the performance and scalability of the MEC system, making it well-suited for use in a variety of edge computing scenarios.

One key aspect of our solution is the inclusion of an off-chain verification system that allows users to verify the integrity of the compute hosts they are using. This system allows users to choose from a variety of providers, each of which has a reputation for providing a certain level of integrity. Additionally, users have the option to compute their code in enclaves provided by the hosts, further enhancing the security of their workloads.

Risks Users Code and Computational Integrity is subjective and relative to the reliability of TEEs, Zero Knowledge Proofs and Secure MPC. Risk of traditional security attacks including cross-site scripting, host spoofing and social engineering must be taken into consideration Smart Contract security including insurance of non-custodial key management is subject to proper advisory and implementation of Lit Protocol's Standards and Structs

In conclusion, we believe that our project has the potential to provide significant value to the decentralized computing ecosystem by enabling users to execute distributed workloads at scale in a secure and decentralized manner. The inclusion of an off-chain verification system and the ability to compute code in enclaves provided by trusted/verified nodes

Deliverables

Deliverable Proposal: POC for Decentralized Edge Compute Cloud

Overview:

The goal of this deliverable is to demonstrate the feasibility of using P2P nodes running on the Hyperswarm network to execute compute functions in a given region, with the provability of function execution being secured by Lit Protocol contracts. This proof-of-concept (POC) will lay the foundation for a decentralized MEC (Multi-Access Edge Computing) system that can securely and reliably execute distributed workloads at scale.

Implementation Details:

P2P nodes on the Hyperswarm network will be responsible for discovering and connecting to other nodes in their region.
Functions to be executed will be compiled into WebAssembly and deployed to the P2P nodes by the function owner.
The function owner will sign the function execution request with their private key, creating a signed message that can be verified using their corresponding public key.
The signed message, along with the WebAssembly code for the function, will be sent to the target P2P node via the Hyperswarm network.
Upon receipt of the function execution request, the target P2P node will verify the signed message using the function owner's public key, which is stored in a Lit Protocol contract.
If the signed message is successfully verified, the target P2P node will execute the function and return the result to the function owner.
The execution of the function, along with the input and output data, will be signed and recorded in a Lit Protocol contract to provide a verifiable audit trail of the function execution.

Technology Stack:

Hyperswarm: P2P networking library for peer discovery and connection management.
Lit Protocol: Public key management contracts for signed message verification and function execution auditing.
WebAssembly: Portable code format for execution in a variety of environments, including P2P nodes.
Firecracker microVMs: Lightweight virtualization technology for securely executing functions with minimal overhead.

Development Roadmap

Timeline:

month  1: Set up Hyperswarm P2P nodes and establish basic connectivity via EVM Wallets.
month 2: Implement signed message verification using Lit Protocol contracts.
month 3: Compile functions into WebAssembly and deploy to P2P nodes.
month 4: Test function execution and signed message verification on P2P nodes.
month 5: Record function execution details in the Lit Protocol contract for auditing purposes.

Upon completion of this deliverable, we will have a POC system that demonstrates the feasibility of using P2P nodes on the Hyperswarm network to execute compute functions in a given region, with the provability of function execution being secured by Lit Protocol contracts. The POC system will allow function owners to deploy WebAssembly code to P2P nodes, which can then be executed by signing and verifying a message with a public/private key pair managed by a Lit Protocol contract. Function execution and input/output data will also be recorded in the Lit Protocol contract for auditing purposes. This POC system will provide a solid foundation for the further development of a decentralized MEC system that can securely and reliably execute distributed workloads at scale.

3 Team Members

Development of P2P node software for Hyperswarm network: $10,000
Integration of Lit Protocol contracts for function execution and key management: $15,000
Implementation of WebAssembly code execution on P2P nodes: $10,000
Integration of Firecracker microVMs for secure function execution: $15,000
Testing and debugging of POC system: $5,000
Documentation, Demonstration, Advocacy/Community Outreach, and technical writing: $10,000

2/1/2023 -> 7/1/2023

Total Budget Requested

Total cost: $75,000

The funds will be used to cover the development and implementation of the POC system, as well as testing, debugging, and documentation efforts. These tasks are necessary to ensure that the POC system is reliable, secure, and easy to use for function owners and P2P nodes.

Maintenance and Upgrade Plans

Team

Dylan Wong - Founder John May - Co-Founder Ryan Wong - Systems Engineer

Team Member LinkedIn Profiles

https://www.linkedin.com/in/dylan-w-998b85116/ https://www.linkedin.com/in/ryan-jia-wong/ https://www.linkedin.com/in/johnbmay/

Team Website

https://l3s.vercel.app/

Relevant Experience

Dylan Wong - Developer of 11 years, Gained over 2.3 Million visits on the ROBLOX Metaverse Platform with Zombie Survival Style First-Person-Shooter games with in-game microtransactions and purchases. Juniper Certified Network Associate, AWS Certified Associate Solutions Architect, previously worked as DevOps/Platform/SRE for the first public Decentralized Oracle Platform/Protocol, currently a Co-Op member at Red Hat in the Telco Solutions Architecture Group focusing on Distributed Computing via Multi-Hybrid Clouds for Telcos and CNF Providers, specifically on Smart Green Cloud Operations within a Distributed System, showcasing the usage of Carbon Credits and Penalties through autonomous metric optimization via Distributed SNMP Metrics Streaming aggregated through Telegraf, Prometheus and Grafana. Specializing in Distributed Cloud Technologies, Amazon Web Services, Ansible Automation Platform, OpenShift, Containers, Go, EVM Based Smart Contracts, and Peer-to-Peer (P2P) Networking via WebRTC and Holepunching techniques and Bit-Torrent Style Distributed Hash Tables and Web Application Development via Node, NextJs, Typescript and Serverless Functions

John May - Currently a Technical Lead within his employer specializing in Rust and Solidity-based ERC721 and ERC1155 mechanisms. Knowledgeable in UTXOs, ZK Proofs, and Secure Multi-Party Computation. Previously awarded multiple hackathons including ConneXT at ETH NYC

Ryan Wong - Developer of 10 years, Amazon Web Services Certified Associate Solutions Architect, and HashiCorp Terraform Associate. Currently, a Student Researcher at The University of California focusing on AI with competency in DevOps, SRE, and Distributed Networks

Team code repositories

Dylan Wong - https://github.com/redhat-partner-solutions/crucible pull request to upstream external partner project at Red Hat for 5G Kubernetes Deployment Automation

https://github.com/fruitbox12/deMEC-Cloud deMEC node operator basic implementation with wagmi UI, P2P Hyperswarm Implementation for UDP Connections through Next.Js Routes and TypeScript (we privately have an implementation that also supports node discovery via Websocket or WebRTC channels for browser peers)

John May - https://github.com/johhonn/zkNFT - NFT Staking for Generating ZK Proofs

Ryan Wong - https://github.com/RyanJWong/twitter-spaces-api-example Twitter Spaces API integration example with NodeJs and Cloudflare Workers for Web 3 Apps

Additional Information

GitHub dylanwong007@gmail.com

One of the major drawbacks of relying on public clouds for web3 infrastructure is the lack of control and customization that users have over their deployment environment. Public cloud providers often have strict rules and regulations in place that limit the ability of users to fully optimize their deployments for specific use cases and requirements. This can result in poor performance, high costs, and other issues that can negatively impact the user experience. In contrast, decentralized edge computing networks like the one proposed in this proposal offer users the ability to fully customize their deployment environment and fine-tune their infrastructure to meet their specific needs. This results in more efficient and cost-effective deployments that are better suited for web3 applications and other demanding workloads. Furthermore, public clouds are generally centralized and controlled by a single entity, which can create security and privacy concerns for users. Decentralized edge computing networks, on the other hand, are distributed and decentralized, which makes them more resistant to attacks and better able to protect user data. Overall, the adoption of decentralized edge computing networks like the one proposed in this proposal is crucial for the long-term success of web3 and the broader decentralization movement. By leveraging the power of these networks, web3 developers and users can build a more scalable, efficient, and secure infrastructure that is better suited to meet the needs of modern applications.

LIT-Protocol / LitGrants