Open Grant Proposal: Storage Aggregator | Files

[lk2684753]

Open Grant Proposal: Storage Aggregator | Files

Name of Project: Storage Aggregator | Files

Proposal Category: core-dev

Proposer: lk2684753

Do you agree to open source all work you do on behalf of this RFP and dual-license under MIT and APACHE2 licenses?: "Yes"

Project Description

An API for unified storage, billions of possibilities!

Storage Aggregator provides a unified storage API, making it easier to retrieve and store billions of blockchain data!

Storage Aggregator provides you with dozens of custom storage platforms and intelligent file retrieval services!

Storage Aggregator Personal Web3 Storage in a Decentralized Web!

The Storage Aggregator Network is a great decentralized storage network that allows anyone involved in storage to be rewarded fairly from the protocol! SA is built on decentralized web and provides a native cross-chain communication tray based on SCMP (Cross-Chain Message Passing), which is stored in the SA storage cloud through a unified data exchange API that is 100% compatible with Amazon S3 API and retrieve user data. This allows any project to use any AWS library or SDK and start using our system without extensive integration work.

NFT support

• Built-in option to tag any uploaded file
• Use ERC-721 or ERC-1115 NFT standard
• Universal digital asset storage for any NFT marketplace

Designed for Defi

• Transparent, immutable and cryptographically verifiable
• Large mixed datasets (like payment schedules and token binding curves)
• Web3-based "all-in-one" experience for unified data management

Hot and cold data

• Smart selection for "cold" & "hot" data based on data type
• "Cold" data is rarely used, but must be kept long-term or even permanently; "Hot" data is used frequently and may update deleted data
• Dozens of times cheaper and faster than normal storage like Amazon Clouds

Full granular visibility

• Storage Aggregator has indexed and decoded major storage platforms to facilitate data storage and retrieval through a unified API: make it available to developers anytime, anywhere. Provides convenience for storing data on billions of blockchains.
• Switch between Storage Aggregators without data migration

Supports multiple storage platforms

• Works with multiple storage platforms (Amazon Glacier, Google Coldline, IPFS, Swarm,

  Simple and seamless integration of Filecoin, storj, arweave, SIQ, 0CHAIN, Lambad, BitTorrent, Alibaba Cloud, etc.)

• Developers use Storage Aggregator to reduce time to market from 2 weeks to a few hours.

Use Case Marketplace

• Developers can create their own API endpoints for their projects and list them on the marketplace for others to use.

No-code solution

• Storage Aggregator does not require developers to write indexing code or understand the nuances of storage platform interfaces. Also accessible to non-technical users.

Primer conversion engine

• An internal transformation language inspired by the MongoDB query language. Using Primer, developers can select only the fields they want and further aggregate the data to meet their needs. All of this is done on the fly at query time.

Decentralized index query middleware

• The Storage Aggregator network has fully decentralized storage and computation layers with low fees and high throughput.

Value

The Storage Aggregator ecosystem has three types of nodes: file nodes provide file upload/download services, gateway nodes work directly with the decentralized cloud, and billing nodes calculate storage costs and charge users. Regarding storage fees, we use Storage Aggregator network tokens, Fil, USDC, etc. for transaction billing (per data storage/retrieval operation)

We have our own storage miner node. We need a large amount of real data storage to replace the 0 data encapsulated in the sector, and through the aggregator, we can obtain more real data to improve the computing power of the node.

Blockchain offers a new frontier of development for businesses, consumers and software developers. Due to its decentralized architecture, immutable ledger and highly secure framework, the application of this technology will impact industries from finance and retail to personal identity and cryptocurrencies. Storage Aggregator provides a unified API to store billions of blockchain data Bring convenience!

Deliverables

We will deliver the following software and documentation artifacts in the scope of this grant application.

• Storage Aggregator official website.
• Storage Aggregator Blockchain Network.
• Provide a unified API to index and decode mainstream storage platforms.
• Primer Transformation Engine, an internal transformation language inspired by the MongoDB query language. Using Primer, developers can select only the fields they want and further aggregate the data to meet their needs. All of this is done on the fly at query time
• Provides a fully decentralized storage and computing layer with low fees and high throughput.
• NFT & Defi support
• Documentation. Tutorials for node operators and developers, as well as contributor guides.

Development Roadmap

This project will require a full 20 weeks to develop a full mainnet Beta. Cost basis is paid out in FIL token or stable coin.

• M1 (Week 1-4) 4 Developers & 1 Product Manager

  • Requirements finalization
  • Storage Aggregator official website UI design
  • Storage Aggregator Blockchain Network Deployment
  • Related functional tests
  • Budget requested: $25,000

• M2 (Week 4-10)5 developers

  • Develop unified storage API to index and decode mainstream storage platforms
  • Mainstream storage platform node construction
  • Develop primer switching engine
  • Write relevant documentation
  • Related functional tests
  • Budget requested: $45000

• M3 (Week 12 to 14)5 developers

  • Payment System Design
  • Storage and Bandwidth Charge Design
  • Develop decentralized index query middleware
  • Front-end development storage file information display and operation
  • Storage permission design
  • Related functional tests
  • Budget requested: $30,000

• M4 (Week 14 to 18) 6 developers

  • Smart Contract Integration(NFT|Defi)

  • Regression testing

  • Bug fixes and Improvements

  • Deploying Application

  • Budget requested: $30,000

• M5 (Week 18 to 20) 4 developers

  • Governance Testing

  • Tokenomics Finalized

  • Payment Model Testing

  • Features/changes will be as a result of DAO proposals.

  • Budget requested: $15,000

Total Budget Requested

Due to the heavy workload, it is necessary to mobilize engineers in and out of the project to keep the progress in sync with the upstream storage platform progress, and the diverse skills required (12 people x month full-time GO/Rust/blockchain engineer/front-end engineer/product manager/UI design ), we are requesting a $145,000 Foundation grant. All funds will be used to cover personnel costs, including wages, taxes, insurance, benefits. The important thing is that we need to buy some machines to deploy the service and we are willing to pay part of the budget in FIL.

We can invoice based on completed milestones.

• Phase 1: $25,000
• Phase 2: $45000
• Phase 3: $30,000
• Phase 4: $30,000
• Phase 5: $15,000

Total :$145,000

Maintenance and Upgrade Plans

Our team is committed to maintaining the maintenance, upgrade and operation of the Storage Aggregator network; in the later stage, based on user feedback, we will improve our project, and the team will open source and maintain the development code for at least three years.

Team

Contact Info

lk2684753@163.com

Team Members

• Liangkai

  Responsible for jingdong cloud hosting and cloud architecture of database, and research and development work, previously in sohu product technology center is responsible for sohu webim, real-time gaming platform technology products such as research and development work with architecture, after joining sina weibo underlying platform research and development work of sina weibo, dominated the redis in sina weibo's development and application of mass customization. Later, I joined Tencent and was responsible for the r&d and architecture of Tencent's large-scale distributed storage project, providing highly reliable storage services for Tencent's various business product lines. In terms of technology, I mainly focus on distributed storage/cache/database, Linux kernel and other basic technical fields, and have a deep understanding of blockchain basic technology；

• Dongjiahong

  Familiar with Golang language development and related tool chain; More than 8 years of basic system development and architecture experience in Internet industry; Experience in large-scale distributed storage/cache/database system design, knowledge of open source architectures such as Ceph,Swift,Sheepdog, etc. Have a deep understanding of web server design, read some excellent open-source project source code, such as Nginx,Redis,Memcached, etc. Blockchain technology enthusiasts, have unique insights into the design of economic models;

• Zoe

  Years of experience in data product development and architecture. I have led the development and community construction of various data products in JINGdong and several companies. I am active in the field of data products and have rich experience in various products in the field of database and big data. At present, I am leading the r&d and promotion of flexible database in Jingdong Mall. Blockchain technology enthusiast, has participated in the development of Lambda and NEO public chain, and is currently committed to the planning and development of NFT market projects;

• Mars

Has a passion for programming and problem-solving and strives to write clean and modular code. Marston is proficient in functional and object-orient programming, allowing him to develop software in a cross-chain environment efficiently. At this time, Marston is fluent in Javascript, Rust, Java, C/C++, Python, PHP, and SQL. Before starting JACKAL, Marston has developed a Web3 digital evidence capture and chain of custody software in the polygon ecosystem. Previously he has also created a Minecraft modification with over 3 million downloads.

• Vivian Hu

Is a DevRel expert and product manager for the project. She is responsible for technical documentation and developer engagement.

Team code repositories

https://github.com/neo-project/neo

https://github.com/lk2684753/go-palletone

https://github.com/lk2684753/chubaofs

https://github.com/tpmccallum

https://www.linkedin.com/in/vivian-hu-513118185/

Additional Information

On storage miners, we currently have 100P+ computing power;

The following are some node addresses for maintenance and operation:

F020330 f0855056 F0711266 F0133235

Wiki working document (subject to change): https://bit.ly/3n4aadH

Discord: https://discord.gg/XKHdCxFkQN

[ErinOCon]

Hi @lk2684753 , Thank you for your proposal. We will review this and get back to you, on this thread, with a status update or questions.

[lk2684753]

Hi @lk2684753 , Thank you for your proposal. We will review this and get back to you, on this thread, with a status update or questions.

Please let us know the progress of the proposal, the team is ready to develop this project. This is our second proposal, before the proposal at https://github.com/filecoin-project/devgrants/issues/375

[ErinOCon]

Hi @lk2684753, thank you for being in touch! We are currently connecting with experts in the ecosystem to review the usability and feasibility of this project. We expect to be in touch with an update soon!

[lk2684753]

Can you bring me up to date? I can update my proposal @ErinOCon

[ErinOCon]

Hi @lk2684753, I expect to have an update available by the end of this week. Thank you for checking in and for your patience!

[ErinOCon]

HI @lk2684753, thank you for your proposal and for your patience as we completed our review. Unfortunately, we won’t be able to fund this project at this time. While it's great in its objective, we recommend focusing on one or two technology stacks (Filecoin and AWS for example) on the development roadmap. We’re happy to reconsider funding for the development of a POC with a stack of choice.