subconsciousnetwork / subconscious

Apache License 2.0
9 stars 0 forks source link

Multiplayer backlinks #323

Closed gordonbrander closed 1 year ago

gordonbrander commented 2 years ago

Story

As a user, when I am visiting a note, I want to see related notes from spheres I follow, so that I have ambient awareness of related thoughts my follows have had on this topic.

gretchenfrage commented 1 year ago

I'm curious if you have thoughts or plans on how to implement this--it's a concept I find particularly vexing.

The way I see it, the fundamental issue here is how to define whether a particular backlink is "relevant." This is, in such a situation:

Should C see B's backlinked post? More generally, if there are say thousands of posts in existence which backlink to A's post, how should they be ranked, and which ones should be cut off in the UI? Especially if bots go out there and start generating everything from potentially-relevant associated information to pure spam.

Probably approaches can be divided into two categories: ones which rely on some objective ranking, and ones which allow for different lenses of viewing to maintain different rankings.

Objective ranking

An objective ranking would entail all nodes reaching consensus on the relevancy (ranking or boolean) of a particular backlink from the perspective of the original post.

Pros:

Cons:

Possible consensus algorithms:

Subjective ranking

A subjective ranking would entail allowing for different lenses of browsing (users, browsers, states or components within browsers) to maintain different perspectives regarding which backlinks and relevant and in what ranking to a particular original post.

Some example definitions for these perspectives could be:

The pro of subjective ranking is epistemic flexibility and whatnot, equal and opposite to the cons of objective ranking.

The con of subjective ranking is that disagreement on the data is fundamentally in tension with peer-to-peer data structures like DHTs. If one is to allow for nodes to disagree on the correct {post id -> [post ids with relevant backlinks]} mapping, they can't exactly deduplicate ranges of that mapping in a conventional DHT for efficiency and robustness and stuff.

The polar opposite way to implement such a thing would be for each node to just maintain a local index of all, or at least as many as it practically can, of the posts it could potentially find relevant as backlinks. So, for example, if a node wants to rank backlinks by degrees of separation of following, it can continuously fetch and index all the posts made by everyone they follow, and everyone one of those people follows, etc, and then consult this database. I think this is most similar to the activitypub style of approaching things.

The downsides of this are this are that it becomes heavy-weight and unscalable. If one is, say, using subconscious on their phone, they probably don't want their mobile app spending its various resources maintaining a constant firehose stream of the whole web. It would become straight-up not possible when the network scales to a certain rate of posting, which would drive users to begin to lose the informational capacity to see beyond their local social circle (unfortunate!). New users would have to wait for an extended period of time for the system to back-propagate, and that would be their first experience with the system.

Hybrid approach

One hybrid approach would be a federation-style approach. "A perspective is too expensive for each individual to have, so have people choose which perspective instance they want to connect to." The more straightforward way to do this would be by maintaining activitypub-style instance servers or server clusters. I worry about the centralizing tendencies to that--after all, email is a federated protocol, but it's become extremely centralized because of the high barriers to running your own email server reliably.

Alternatively, it's conceivable that if these perspectives could be expressed in some language capable of doing zero-knowledge proofs, then nodes of a particular perspective could form their own DHT with each other for it. This would be hard. It would also still be concerning in how it still leaves non-epistemic energy barriers against dissent.

I think the sort of north star platonic ideal of such peer-to-peer associative structures would be a system that uses DHT-reminiscent deduplication where overlaps in perspectives occur while still fluidly allowing partial disagreement. Unfortunately I am not aware of the existence of such a technology, and my attempts to research it have left me only with fear at the looming suspicion that it has something to do with category theory.