ICPHackathon / ICP-Hackathon-2023

1 stars 0 forks source link

Proton - A Decentralized Social Media Totally on Chain #4

Open NashAiomos opened 6 months ago

NashAiomos commented 6 months ago

Proton - A SocialFi DApp totally base on Actor model


Product Introduction 👇

Slow is fast. Decentralization.

Now we are designing a decentralized social media called Proton based on the Actor model. First we need to understand the basic principles of the Actor model. The Actor model is a concurrent computing model that achieves parallel and distributed computing through message passing and asynchronous processing.

So when designing a DApp, each Canister should be responsible for a different functional module. For example, some Canisters are responsible for recording user profiles, some for storing posts. Also, pay attention to the scalability of the Canisters. We can dynamically create Canisters of the same type to handle high loads.


Design Philosophy

We want to build a truly open Web3 DApp that is modular and data sovereign based on the Actor model.

We hope that each user has their own independent space (Feed Canister) that they fully control. Users can even directly deploy their own independent Feed Canister with code to interact with Proton. (This is cumbersome, only suitable for programmer users, who can develop advanced custom features for the container.) This allows the community to create custom advanced features.


Idea

First there is a scalable public area that receives posts from all users. There is also a user area that records user registration, personal profiles, and following relationships.

We create a Feed for each user to store their own information flow. The Feed is also the user’s private space. Users can store posts in their own Canister (Feed). Except for themselves, no one else can delete it.

The interaction between users and the public area is automatically handled by the Feed Canister. Users only need to query their own Feed to access the latest information stream they are following. Posting, commenting, and liking interactions are also automatically completed by Feed after the initial action.

Users can also add some advanced custom features by deploying their own independent Feed to interact with the public area. For example, only send posts point-to-point to some Feeds, establishing private small social circles; or only connect AI for automatic posting, etc. Any function can be implemented. The community can develop secondary and freely expand various functions. For example, adding a point-to-point private messaging feature.

This is a completely open, slightly slower decentralized application. This design sacrifices some speed for decentralization, just like Bitcoin.

The advantage is that the user’s front end only needs to query their own Feed to get posts from people they follow. It’s convenient and fast. Everything in the background is completed by decentralized collaboration between Canisters, completely decoupled. If a few Canisters go down, it does not affect the continued operation of this system. (If Fetch goes down, you can create a few more)

If the system cannot be recovered temporarily, the Feed can send posts in batches directly to followers using ignore call. That is, two post delivery processes are built into the Feed: posting through the Fetch relay station, and point-to-point posting.


Message Transmission Process

When a user posts, the Feed first stores the post in its own information flow, and then sends the post point-to-point to the follower’s Feeds and the public area according to the followers list. But what if there are 10,000 followers? The situation is not so good, because message sending and receiving between Canisters is limited by max input/output queue size, and cannot send so many at once. It would take the Feed a long time to send them all in batches.

To increase throughput, we add a message relay station: Fetch. The Feed first sends the post to the public area, then sends the post ID and followers to Fetch. Fetch records it, and then notifies these followers’ Feeds according to the algorithm to fetch which posts, and finally the Feed fetches the posts from the public area.

This way, even with many followers, they can fetch posts from the public area in turn under the coordination of Fetch.


Automatic Pressure Adjustment

After Fetch accumulates a certain number of messages, it adjusts the notification order and interval through algorithms (which Feed to notify first and how many milliseconds to wait before notifying other Feeds after each notification), to ensure the query pressure on the public area cannot be too high. When the Feed receives the notification, if the fetch fails, it should wait 20 seconds before trying again.

If the public area faces too much query pressure, it will tell the Root Fetch: “Slow down notifying Feeds”, and the Root Fetch will notify the Fetches under it to reduce notification frequency. If the pressure remains high after 10 minutes, the Root Post can also create a new Bucket to send new posts to.

Also, if the number of users increases, Fetch itself can also be added as needed.

Because every time a Feed sends a message, Fetch has to notify many other Feeds. So in this open environment, it is easy to cause Dos attacks. Therefore, when the Feed sends information to Fetch, it needs to pay a Gas fee. Only after receiving the Gas fee will Fetch put the information into the “pending notification” list.

The Gas fees for posting, liking, and commenting are 100000000, 10000000, and 1000000 respectively.

However, these are plans for the future. For now, let’s design the overall framework first and continue to optimize the details later.

We put the architecture at the end because it is too long~


Product Preview Image

image

image

Presentation Slides

https://docs.google.com/presentation/d/1wNND_4PEwpPQtgMtN2IB_A7ehtONXOrQaZdDDl9EpJM/edit?usp=sharing

Code URL

https://github.com/NeutronStarDAO/Proton

Video demo

Promotional Video

Deployed URL

Team Information

Two years ago, we began learning DApp development on the Internet computer, back in November 2021. After two years of continuous learning, we now have a deep understanding of the underlying principles of the Internet computer. We have launched a book to introduce the underlying technology of the Internet computer, covering architecture, formulas, protocols, ChainKey, cryptography, and more.

You can check it out here: https://neutronstardao.github.io/constellation.github.io

Now, we are planning to create a social DApp: Proton~ By Core Engineer @xiaoyuanxun, @Shuaige1234567 And @NeutronStarPRO .


Architecture

The architecture is based on a distributed peer-to-peer Actor model with a push-pull design. Overall, you can divide Proton into four modules: User, Feed, Post, and Fetch.

Users can follow/unfollow others, view the latest public posts (from everyone), see their feed (their own and followed users’ posts), create posts, repost, comment, like, favorite/unfavorite.


User

The user area records user information and relationships, like profiles and follow relationships.

The User canister stores basic user info - UserId, name, company, school, occupation, bio, follow relationships, their Feed canister ID, etc.

Users can call functions here to follow someone, update their profile, or check who they follow and someone else’s relationships.

When a user follows someone new or gets a new follower, their Feed is notified to update the list.


Post

The public area stores all publicly posted content. The Root Post can create many Bucket canisters to store posts.

Root Post

The public area stores all public posts. The Root Post can create many Bucket canisters to store posts.

Root Post can create Buckets, check available Buckets, list all Buckets, and list full Buckets.

Root Post starts by creating 5 Buckets. When one fills up, it creates a new one, keeping 5 available.

When the front-end first loads, the Feed immediately queries Root Post for an available Bucket. Root Post randomly returns one. The Feed stores the “available Bucket” ID, updating it when queried.

Call Bucket to query the latest 5 posts to show latest public posts. When a user posts, call Bucket to store it. When a user’s Feed gets a bunch of post IDs from Fetch, it can then provide those post IDs to the Bucket to query for the actual posts.

Bucket

Buckets can store and query posts.

There are 3 query functions - total posts, get specific posts by ID (up to 7 at once), and latest n posts.

When querying specific or latest posts, it returns post details and current likes/comments.

Buckets receive new posts, comments, and likes. It checks for duplicate IDs before accepting a post.

It notifies Comment Fetch and Like Fetch about post IDs with new comments/likes.


Feed

The information feed stores personal feeds for each user. The Root Feed creates a Feed for every user.

Root Feed

Root Feed is responsible for creating a personal canister for each user and tracking the total canisters created and their IDs.

Feed

Users interact with Proton through their Feed - viewing, posting, commenting, liking, etc.

A user’s Feed stores their followers (for pushing posts, comments, likes), following (for receiving posts), feed (last 3000 posts only), and saved posts (up to 500 posts).

Each post has a timestamp, poster’s UserId, PostId, and RepostId (empty if not a repost).


The PostId is the Bucket canister ID + UserId + increment. This allows direct post ID creation without communicating with the Bucket.

For example: aaaaa-aaa-bbbbb-bbb-1, aaaaa-aaa-bbbbb-bbb-2, aaaaa-aaa-bbbbb-bbb-3…


Querying posts:

There are 3 functions to query Feed posts - total posts, get post by ID, and latest n posts.

Posting:

When user A posts, the frontend sends the post to their Feed.

The Feed stores the new post.

Then it sends the post to the public Bucket and notifies Fetch with the poster, post ID, and followers C & D.

image-20231117004133644-1700192647376-8.png

Fetch records this and individually notifies C & D’s Feeds to pull the post by ID. After notifying, Fetch deletes the records.

image-20231117004213850-1700192647376-10.png

When Feed gets the post IDs to pull, it adds the corresponding posts from the public Bucket into the feed stream. (Here, User C pulls posts 1, 6, 7, 15)

image-20231117005218488-1700192647376-11.png

When Users C & D query their Feed, they see Poster A’s new post right away.

If User E later follows A, their Feed only receives A’s new posts going forward.

The frontend only sends one request. Further pushes (like notifying the public Bucket) are handled by the canisters.

image-20231120171812860.png

With more users, one Fetch may get overwhelmed with posts. More Fetches can be added horizontally as needed.

Deleting posts:

Posts cannot be deleted in the followers' feed canister.

Currently, considering the private nature of the feed canister, it is challenging to delete already posted messages.

This difficulty arises because the posts are stored dispersedly in various feeds, and since feeds are controlled by others, the owner of the feed can refuse deletion requests.

Additionally, for posts in public area, deletion is, of course, possible. After future DAO control, there may be the option for collective voting to remove certain posts.

Reposting:

When User C reposts Post 15 to followers H, I, J, K: Reposter: C, Post ID: post15_id, Followers: H, I, J, K is sent to Fetch.

Fetch records and notifies H, I, J, K’s Feeds, which pull Post 15 by ID.

When User C reposts, the original poster A remains the same. Only the reposter is User C.

Commenting:

Similar to posting, handled by Comment Fetch.

Anyone (User X) can comment. The public Bucket is notified of the comment with the post ID.

Comment Fetch gets the post creator’s followers from User. It rejects if the creator isn’t found.

It adds the post ID, creator, and followers to its “to notify” queue.

image-20231129154312702.png

After Comment Fetch notifies followers, they query the post’s comments from the public Bucket and update their Feeds.

If a follower D reposted the post, their Feed further notifies Comment Fetch when receiving a new comment.

Comments cannot be deleted.

Liking:

Similar to posting, handled by Like Fetch.

Anyone (User X) can like a post. The Bucket notifies Like Fetch of the new like with the post ID.

Like Fetch gets the post creator’s followers from User. It rejects if the creator isn’t found.

It adds the post ID, creator, and followers to its “to notify” queue.

image

After Like Fetch notifies followers, they update the post’s like count in their Feeds.

If a follower D reposted the post, their Feed further notifies Like Fetch about new likes.

Likes cannot be deleted.


Fetch

Receives all posting, commenting, and liking notifications and forwards them to relevant Feeds.

Root Fetch

Root Fetch dynamically creates multiple Fetch canisters - Post Fetch, Like Fetch, Comment Fetch. It can also list available Fetches.

Post Fetch

Receives: Post ID, poster, reposter, followers, cycles.

Maintains a table of posts to notify each user about.

Uses an algorithm with ignore calls to notify followers’ Feeds in batches.

Comment Fetch

Receives new comment notifications from Buckets: Post ID, poster (A), reposter (empty).

Gets the poster’s followers from User.

Maintains a table of posts to notify about.

Uses an algorithm with ignore calls to notify followers’ Feeds in batches.

If a follower C reposted the post, their Feed further notifies Comment Fetch when receiving a new comment.

Like Fetch

Similar to Comment Fetch, for likes instead of comments.

The core User, Post, Fetch, and Feed modules make up Proton’s architecture.

Beyond this, Feeds can communicate directly for more features…

About Forum Link

About the Future

Everything is up to the users' choice.

Users have the option to privately send posts to selected followers through Feed, maintaining confidentiality without public disclosure.

They can also establish their own "community server," essentially a self-controlled public space for the community. In this setup, people's Feeds can subscribe to the community service area, requesting updates from the server every 2 hours. The Feed contains the post ID from the last update; when making a request, the Feed provides the previous post ID, and the community server retrieves and returns posts since the last update.

image-20231220175917344

The entire system is modular, allowing communities to autonomously build various components. For example, users can create their own community server, broadcast matrix, and more (the broadcast matrix will be explained later).

Broadcast Matrix

Now, let's consider a scenario with high concurrency. Suppose Fetch needs to notify 10,000 Feeds, but due to the Canister system's message queue limit, let's assume Fetch can only send notifications to 500 Feeds at a time.

If only one Fetch is working, it would take 20 rounds to complete the notifications. This is where multiple Fetch instances come into play.

One Fetch first divides the list of users to notify into 5 parts and distributes them among 5 Fetch instances. This way, 2500 notifications can be sent at once, and the entire push can be completed in just 4 rounds!

image-20231224001103346

The Canister's message output queue is set at 500. Therefore, we can establish relay stations with different sending capacities: 500, 2500, 5000, 10000, and so on...

One Canister can send 500 messages at a time. Five Canisters can send 2500 messages, ten Canisters can send 5000 messages, and twenty Canisters can send 10000 messages in one go.

image-20231201010601827

For safety, it's advisable to leave some message output to ensure normal communication and to call the Root Fetch in emergencies. Thus, it's recommended to set a maximum of 430 notifications per Fetch cycle.

To prevent DoS attacks, users should include some Cycles with each post, comment, or like. Different types of broadcast matrices incur different charges.

If someone has a large number of followers, they can choose not to trust anyone else and establish a private broadcast matrix exclusively for notifying their own fans.