filecoin-project / devgrants

👟 Apply for a Filecoin devgrant. Help build the Filecoin ecosystem!
Other
371 stars 308 forks source link

Open Grant Proposal: Meson Network #839

Closed bitruss closed 1 year ago

bitruss commented 2 years ago

Name of Project: Meson Network

Proposal Category: devtools-libraries

Proposer: bitruss

Do you agree to open source all work you do on behalf of this RFP and dual-license under MIT, APACHE2, or GPL licenses?: Yes

Project Description

Meson Network is committed to creating an efficient bandwidth marketplace on Web3, using a blockchain protocol model to replace the traditional labor-based sales models, consolidating and monetizing idle bandwidth from long-tail users at a low cost. Meson Network is the foundation of data transmission for decentralized storage, computation, and the emerging Web3 Dapp ecosystem.

Meson Network is committed to optimizing the speed at which the Filecoin/IPFS public gateway distributes reads. At present, the mainstream public gateways of Filecoin/IPFS are ipfs.io, cloudflare-ipfs.com, and dweb.link.

ipfs.io is the most frequently used gateway and the only node that can access files over 500MB, but its operating efficiency in global acceleration is not dominant. cloudflare-ipfs.com is relatively fast, but cannot speed up large files. dweb.link is currently the recommended gateway, but it has the same problem with ipfs.io that it is not as fast as one might expect. In view of this, Meson Network is committed to providing Filecoin/IPFS users with a secure, efficient, and usable decentralized gateway option with the help of decentralized nodes distributed around the world.

https://docs.meson.network/using/#example-meson-enhances-ipfs

Value

  1. To support the Filecoin/IPFS gateway service achieve decentralization, the addition of Meson can alleviate the current situation that the bottom layer of Web3 information transmission is surrounded by centralized gateways.
  2. At present, except ipfs.io, other gateways cannot retrieve files larger than 500MB, and Meson's gateway can support retrieving files of about 1GB at most.
  3. Given that IPFS will support FFmpeg in the coming future, and the bandwidth consumed by video is very huge, Meson can distribute services better than centralized public gateways.

Deliverables

  1. Provide a decentralized public gateway for Filecoin, and provide CDN services for Filecoin by Meson nodes from around the world.
  2. Meson will be integrated with web3.storage & nft.storage as an HTTP API development tool. Enable Filecoin/IPFS developers to store files to web3.storage & nft.storage with one click through HTTP API development tools on and returns a link to the Meson-accelerated file.
  3. Recommend Filecoin/IPFS to more developers to optimize the development experience affected by bandwidth issues.

Development Roadmap

Phase 1(50,000 USD)

  1. Meson's services better adapt to pictures, texts, web pages, and videos in ipfs
  2. Optimize and support ipfs, ipns file acceleration process
  3. Meson's decentralized Filecoin/IPFS public gateway released
  4. Complete the HTTP API link between Meson and web3.storage & nft.storage

Phase 2(25,000 USD)

  1. Optimize the speed of IPFS reading and caching
  2. Meson supports more IPFS acceleration scenarios
  3. The number of nodes extended to IPFS and Filecoin ecological services

Phase 3(25,000 USD)

  1. Simplify the process of using Meson acceleration services for IPFS and Filecoin users
  2. Fix the problems encountered in the use of the Filecoin/IPFS gateway
  3. Provide long-term and stable services for the Filecoin/IPFS public gateway, so that more IPFS ecological projects can use Meson's decentralized gateway, and also provide Meson ecological projects based on [web3.storage](https:// Decentralized storage options for web3.storage & nft.storage.
  4. Expand the distribution area of nodes to ensure that Meson can provide a good access experience for Filecoin/IPFS users on all continents; expand the ecosystem and introduce more users into the Filecoin/IPFS ecosystem.

Phase 4(45,000 USD)

  1. Accelerated gateway service for IPFS fast indexing in FFmpeg.
  2. Providing bandwidth and technical support for IPFS Compliant HLS Video Streaming.
  3. Providing bandwidth and technical support for IPFS-compatible WebRTC.
  4. Developing a Decentralization Streaming Engine that supports IPFS.

Filecoin Green(55,000 USD)

  1. Helping decarbonize the Filecoin network by contributing to the Data Center Carbon Credit Protocol.
  2. Developing peripheral system services for the Filecoin network, and make greater use of idle resources on Filecoin.
  3. Utilizing idle bandwidth resources to provide green bandwidth resources for the Filecoin network.

Total Budget Requested

$200,000

Maintenance and Upgrade Plans

Team

Team Members

https://meson.network/about.html

Team Member LinkedIn Profiles

https://www.linkedin.com/company/meson-network

Team Website

https://meson.network/

Relevant Experience

The Meson testnet 2.5 is already delivering 30TB+ of traffic per week to the fully decentralized IPFS gateway.

Traffic distributed by Meson Network for IPFS

IPFS public gateway CDN Analysis

Analyzing the centralized CDN service being used by the IPFS public gateway and use large files for accelerated testing.

Large files are tested with Erinnerungen an die Zukunft-the file size is 300MB.

Gateway CDN Transfer large files Url
ipfs.io bunny yes link
gateway.ipfs.io bunny yes link
dweb.link bunny yes link
cloudflare-ipfs.com cloudflare yes link
cf-ipfs.com cloudflare yes link
ipfs.fleek.co cloudflare yes link
ipfs.eternum.io cloudflare no link
ipfs.eth.aragon.network hetzner no link
crustwebsites.net h5 data centers yes link
hardbin.com digitalocean no link
via0.com cloudflare no link
ipfs-gateway.cloud cloudflare yes link
ipfs.telos.miami digitalocean no link
jorropo.net scaleway no link
ipfs.runfission.com amazon no link
nftstorage.link cloudflare yes link
gateway.pinata.cloud digitalocean yes link
4everland.io google yes link
ipfs.best-practice.se mdfnet.se no link
meson.network meson.network yes link

Benchmark Report

Meson Network

Team code repositories

https://github.com/daqnext

Additional Information

ErinOCon commented 2 years ago

Hi @bitruss, thank you for your proposal! We will be in touch once we have completed our initial review.

ErinOCon commented 1 year ago

Hi @bitruss, thank you for your patience! We would like to proceed with a grant that supports the first two phases of your project. For next steps, can you provide a development roadmap that focuses on these to phases? Please note that each milestone should include a description of deliverables, an installment amount, and an estimated timeframe.

ErinOCon commented 1 year ago

Hi @bitruss, I hope you are doing well! Would you be interested in submitting a reworked proposal for this project? Please let me know of any questions.

bitruss commented 1 year ago

Hi @ErinOCon, Sorry to have kept you waiting. Here are our updated Deliverables.

Please let us know if our reworked proposal If the reworked proposal is still incomplete.

Development Roadmap

Phase 1(50,000 USD)

  1. Meson's services better adapt to pictures, texts, web pages, and videos in ipfs
  2. Optimize and support IPFS, IPFS file acceleration process
  3. Meson's decentralized Filecoin/IPFS public gateway released
  4. Complete the HTTP API link between Meson and web3.storage & nft.storage

Phase 2(25,000 USD)

  1. Optimize the speed of IPFS reading and caching
  2. Meson supports more IPFS acceleration scenarios
  3. The number of nodes extended to IPFS and Filecoin ecological services

Deliverables

Phase 1 -
Funding 50,000 USD Duration 1.5 Months (1st. Oct-15th. Nov)
Milestone Area Deliverable Status
A Spec Begin collecting feedback from the community on what kind of metrics they would like to see developed to ease their data-onboarding efforts Active
A Spec adapt to pictures, texts, web pages, and videos in ipfs Active
A Spec services support IPFS, IPFS address Active
B Impl create a dedicated account on meson.network and top up -
B Impl decentralized Filecoin/IPFS public gateway released -
B Impl complete the HTTP API link between meson and web3.storage & nft.storage -
B Impl to test and verify the IPFS integration -
B Doc release full documentation on the decentralized gateway along with practical tutorials -
C Spec collect community feedback on more metrics they would like to have after their initial tryouts -
C Impl revisit each component and add new metrics that the community deemed necessary -
Phase 2
Funding 25,000 USD Duration 1.5 Months (16th. Nov-17th. Dec)
Milestone Area Deliverable Status
A Impl optimize the speed of IPFS reading and caching -
A Spec supports more IPFS acceleration scenarios -
A Spec simplify the process of using Meson acceleration services for IPFS and Filecoin users
A Spec IPFS ecological projects can use meson's decentralized gateway
B Impl expand the ecosystem and introduce more NFT platforms to use IPFS storage and meson gateway -
B Impl to integrate IPFS with ENS domains -
B Impl develop Filecoin/IPFS compatible miner nodes -
B Impl to test and verify the Filecoin/IPFS integration -
B Doc release full documentation on the gateway services along with practical tutorials -
C Impl fix the problems encountered in the use of the Filecoin/IPFS gateway -
C Impl the number of nodes extended to IPFS and Filecoin ecological services -
C Spec collect community feedback on more metrics they would like to have after their initial tryouts -
C Impl revisit each component and add new metrics that the community deemed necessary -
D Impl maintenance and upgrade gateway -
ErinOCon commented 1 year ago

Hi @bitruss, thank you for the updated milestone breakdown! I will be in touch by email to confirm next steps.

bitruss commented 1 year ago
Phase 2
Funding 25,000 USD Duration 1.5 Months (16th. Nov-17th. Dec)
Milestone Area Deliverable Status
A Impl optimize the speed of IPFS reading and caching -
A Spec supports more IPFS acceleration scenarios -
A Spec simplify the process of using Meson acceleration services for IPFS and Filecoin users
A Spec IPFS ecological projects can use meson's decentralized gateway
B Impl expand the ecosystem and introduce more NFT platforms to use IPFS storage and meson gateway -
B Impl to integrate IPFS with ENS domains -
B Impl develop Filecoin/IPFS compatible miner nodes -
B Impl to test and verify the Filecoin/IPFS integration -
B Doc release full documentation on the gateway services along with practical tutorials -
C Impl fix the problems encountered in the use of the Filecoin/IPFS gateway -
C Impl the number of nodes extended to IPFS and Filecoin ecological services -
C Spec collect community feedback on more metrics they would like to have after their initial tryouts -
C Impl revisit each component and add new metrics that the community deemed necessary -
D Impl maintenance and upgrade gateway -

Filecoin/IPFS integration

Meson has achieved a seamless integration with Filecoin/IPFS through Meson Cloud, significantly improving the speed and experience of retrieving and storing Filecoin/IPFS files. Unlike traditional shared upload and retrieval gateways, Meson Cloud adopts completely independent upload and retrieval gateways, reducing the risks associated with single point of failure.

Meson Cloud

Meson Cloud is a Web3 application layer that utilizes Meson's bandwidth marketplace as its foundation.****

mCloud

mCloud is a Web3 Platform-as-a-Service (PaaS) solution that offers a more environmentally sustainable method of hosting and deploying applications. It supports both full-stack applications and docker deployment techniques.

Easily deploy complex architectures with a single click and obtain a secure, fully-constructed hosting infrastructure within seconds.

Users can deploy applications both in traditional data centers and on various blockchain nodes simultaneously, preventing service unavailability due to a single point of dependency.

Maximize the value of your resources by choosing the most cost-effective and affordable service from a range of plans.

https://docs.meson.network/assets/mCloud.87f51039.svg

Gateway X

Gateway X, provided by Meson Cloud Services, is a Gateway Service that functions as a bridge between Web3 such as IPFS/Arweave and traditional Web2.

Gateway X offers users unlimited storage capacity for caching and charges based on bandwidth usage.

Gateway X provides a dedicated gateway connection, supports multiple Web3 Domain Name Service resolution acceleration services, and facilitates seamless interaction with the conventional Internet.

Gateway X offers one-click migration / backup for IPFS & Arweave files. It allows easy pinning of IPFS & Arweave stored files to both Meson Cloud and the user's local storage, guaranteeing the permanent storage of NFTs and other files.

https://docs.meson.network/assets/GatewayX.5f83e2eb.svg

How can Gateway X be distinguished from other hosted services that employ IPFS and Arweave in terms of advantages?

Gateway X is a web3 storage solution that provides several advantages over other hosted services that utilize IPFS and Arweave. One of the most significant benefits of Gateway X is its scalability. The platform is designed to ensure robust performance and dependability, even when dealing with vast amounts of data, thus offering users the ability to rely on it both currently and in the future.

Moreover, Gateway X allows for the backup of IPFS data in Filecoin storage transactions, providing secure storage and facilitating easy retrieval. In contrast, Arweave data is stored permanently and offers encrypted proof of storage, thus enhancing the security and reliability of the stored data.

In addition, Gateway X leverages high-performing infrastructure, including the Meson Network, to offer top-quality IPFS/Arweave implementations. The platform's IPFS/Arweave HTTP gateway boasts a higher number of points of presence (POPs) and cache capacity than many other public gateways, further improving the overall performance and reliability of the service.

GatewayX Pinata Web3.storage Filebase
Local Storage Support yes no no no
Exclusive Upload Gateway yes no no no
Total PoPs 30K+ 200+ 200+ 200+
Total Files pinned Unlimited Limited Limited Limited
Requests Unlimited Limited Limited Limited
Dedicated Gateway Bandwidth Unlimited Limited Limited Limited
Replicated onto Filecoin yes no yes no
IPFS/Arweave Support yes no no no
File Cache 1G 512 MB 512 MB 512 MB

Datacap

Meson will synchronize the data uploaded by users to IPFS in Meson Cloud to Datacap.

https://github.com/filecoin-project/filecoin-plus-large-datasets/issues/1119

ETH.Radio

ETH.Radio is a high-availability Ethereum Name Service (ENS) gateway. It represents a significant step towards the adoption of the decentralized web (dweb) by offering a new way to access ENS resolvable domains. Prior to the launch of ETH.Radio, services such as Cloudflare’s eth.link and the community-driven eth.limo were the primary means of bridging the gap between Web2 and Web3.

With ETH.Radio, users can access ENS domains through a reliable and decentralized gateway that provides an alternative to the centralized services offered by other providers. By leveraging the power of IPFS and ENS, ETH.Radio enables users to access content and services hosted on the dweb in a more resilient and censorship-resistant manner.

Overall, ETH.Radio is a significant development in the ongoing evolution of the dweb, and it offers users a powerful new tool for accessing the decentralized internet.

https://miro.medium.com/max/1400/1*y_MZciCC9Q5IDnyyUa_BjQ.png

IPFS/ENS Gateway Use Cases

Web3 Website

Taking Staking Launchpad as an example, users can add an IPFS link to the ENS Content Hash of depositcontract.eth, which enables the depositcontract.eth domain name to directly redirect to the official website.

https://depositcontract.eth.radio
==>>
https://launchpad.ethereum.org/en/

https://miro.medium.com/max/1400/1*P9vSSe3HnpkOwtI8cxaJTA.png

Code example

<!doctype html>
<html lang="en">

  <body>
    <script>document.location = "https://launchpad.ethereum.org/";</script>
  </body>

</html>

Web3 Gaming

More impressively conquest.eth is accessible via the ETH.Radio gateway : https://conquest.eth.radio/

conquest.eth is an unstoppable and open-ended game of strategy and diplomacy running on ethereum.

https://miro.medium.com/max/1400/1*uS_hjquHo_JRZHhLBNOZIw.png

mCloud Tutorial

Gateway X is a gateway for IPFS/Arweave that utilizes the Meson Cloud infrastructure.

Create Gateway X

https://dashboard.meson.network/labs/gatewayx

Click Gateway X, click on New Gateway to start creating a new one.

https://docs.meson.network/assets/gatewayx-01.4046cfe8.png

Select gateway type, choose IPFSArweavemCDN, select the geographical location for the cloud, set tags, click create, and the creation process will be completed.

https://docs.meson.network/assets/gatewayx-02.1c28824b.png

After waiting for 1-3 minutes, the Gateway will be created and completed.

https://docs.meson.network/assets/gatewayx-03.f9c46a9f.png

Click Edit, click on Switch Default, change Default to Yes, and click Update to confirm.

https://docs.meson.network/assets/gatewayx-04.683bf98d.png

Upload Files

Click Uploading, select Uploading + Local or Uploading + Cloud.

https://docs.meson.network/assets/gatewayx-05.3cd5409f.png

Open the client and use it to start uploading.

https://docs.meson.network/assets/gatewayx-08.8d3436b2.png

https://docs.meson.network/assets/gatewayx-06.d8610c8d.png

Meson Cloud Client

https://github.com/daqnext/meson-cloud-client https://github.com/daqnext/meson-cloud-desktop https://github.com/daqnext/eth.radio

https://docs.meson.network/mcloud/gatewayx.html

Expanding the Filecoin ecosystem

https://twitter.com/NetworkMeson/status/1631214432809533440

https://medium.com/meson-network/matters-lab-launches-collaboration-with-meson-network-f0d84973ad37

https://twitter.com/Mises001/status/1623529479317622786

https://twitter.com/NetworkMeson/status/1702182229307424839

ErinOCon commented 1 year ago

HI @bitruss, thank you for submitting this update! Do I understand correctly that you are one deliverable away from completing phase two? Once this project is complete, please be in touch by email at grants@filecoin.org.