Decentralizing the Tuplespace

[PatrickMockridge]

Speaking with Jim Whitescarver at the moment. He believes that other forks are working on decentralization of the Tuplespace, but that it's important to add it to the issue list here.

This is a necessary and important performance enhancement feature.

[leithaus]

Dear Rho Enthusiasts,

It is encouraging to see people wanting to make improvements. However, some changes will not actually improve things. You cannot decentralize RSpace. You can compose RSpaces, but there is a level below which you cannot decompose an RSpace and still have transactional integrity relative to a given collection of bonded validators.

The whole point of using namespaces and mapping namespaces to aggregated RSpaces is to guarantee validated transactional semantics across a tree of shards. There’s no unit smaller than an RSpace validated by a bonded validator set. There are larger units, but not smaller ones.

In fact, you can calculate the minimum number of names served by a single RSpace shard for it to be viable. Hint: you must have at least the number of wallets as there are validators. But if you only serve that many names, the shard does no useful work.

The biggest issue with this architecture, however is that you can prove, using grade school math, that tokens have a time-to-live. They expire. If they are not expired, they represent a debt the network owes rather than an asset.

Love to all Beings,

—greg

Hope is beyond reason and Love is even greater than that.

On Tue, Feb 20, 2024 at 06:58 Patrick Mockridge @.***> wrote:

Speaking with Jim Whitescarver at the moment. He believes that other forks are working on decentralization of the Tuplespace, but that it's important to add it to the issue list here.

This is a necessary and important performance enhancement feature. Introduction/Motivation/Abstract Motivating Examples Examples Comparison Examples Counter-Examples Design Counter-Examples Drawbacks Alternatives References

— Reply to this email directly, view it on GitHub https://github.com/rchain/rchip-proposals/issues/53, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAASW2WMI3OE4M64INIJQ53YUS22VAVCNFSM6AAAAABDRKB3L6VHI2DSMVQWIX3LMV43ASLTON2WKOZSGE2DINJXHEZDQMY . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[jimscarver]

The whole point of using namespaces and mapping namespaces to aggregated RSpaces is to guarantee validated transactional semantics

You are perfectly correct, decentralizing Rspace across validators is a non starter. That's the domain of sharding. What I am suggesting is is a concept that I believed @tgrospic and yourself were already advocating. Perhaps it was wishful thinking on my part :)

What I am suggesting is hierarchical name space with independent tuple spaces rather that one that grows without bound.

Each name space corresponds to a markov blanket with an interface to an environment. Wallets occupy a global context but I would expect most contracts either do not reference wallets or only reference them through an interface. The context of a contract need not occupy the global tuple space. At any one time I would expect only a relatively small subset of name spaces would be active and there is no need to search them all.

I understand where may be technical challenges with this but I believe we will ultimately need name spaces with an organic nature.

[leithaus]

Dear Jim,

Thank you for clarifying. That was already part of the original design. Mike Stay, Nash Foster, Kyle, and i all agreed a tree of shards with corresponding hierarchical namespaces.

Love to all Beings,

—greg

Hope is beyond reason and Love is even greater than that.

On Sun, Feb 25, 2024 at 12:42 Jim Whitescarver @.***> wrote:

The whole point of using namespaces and mapping namespaces to aggregated RSpaces is to guarantee validated transactional semantics

You are perfectly correct, decentralizing Rspace across validators is a non starter. What I am suggesting is is a concept that I believed @tgrospic https://github.com/tgrospic and yourself were already advocating. Perhaps it was wishful thinking on my part :)

What I am suggesting is hierarchical name space with independent tuple spaces rather that one that grows without bound.

Each name space corresponds to a markov blanket with an interface to an environment. Wallets occupy a global context but I would expect most contracts either do not reference wallets or only reference them through an interface. The context of a contract need not occupy the global tuple space. At any one time I would expect only a relatively small subset of name spaces would be active and there is no need to search them all.

I understand where may be technical challenges with this but I believe we will ultimately need name spaces with an organic nature.

— Reply to this email directly, view it on GitHub https://github.com/rchain/rchip-proposals/issues/53#issuecomment-1963055371, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAASW2T6NQEU3RKRU53IRHLYVOO2DAVCNFSM6AAAAABDRKB3L6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRTGA2TKMZXGE . You are receiving this because you commented.Message ID: @.***>

[jimscarver]

tree of shards

Not sure why you refer to "shards". More clarification may be needed. I am talking about a complete tree in every validator.

It occurs to me that if this is accomplished it might also allow a significantly more effective means for block merge.

[leithaus]

Dear Jim,

Each shard is a collection of validators serving a given namespace. Let us use the notation ( V, N ) where V = { V1, …, Vn } is the validator set and N is some namespace formula. We use accessor notation, V(S) and N( S ) to pick out the components of a shard, S.

Given two namespace formulae, N1 and N2, let’s say that a name, n1, is a prefix of a name n2 just when n1 |= N1 and n2 |= N2 and N1 <= N2 in the evident order. Dually, n2 is an extension of n1.

Additionally we say S1 is a parent of S2 when N( S1 ) <= N( S2 ) and S1 validates all the prefixes of the names that S2 would otherwise validate; and, S2 validates the extensions that satisfy N( S2 ) of the names S1 validates. Note that for transactions involving names in two extensions of S1, say S2 and S3, S1 is the final arbiter if transactional integrity.

While Vi may be in V( S1 ) /\ V( S2 ), it must have a separate bond. So, for convenience we may assume that V( S1 ) /\ V( S2 ) = {}.

This is a generalization of directories and files. Note that there must be a contract governing the exchange rate between tokens in S1 and tokens in S2.

This all gets much more complicated when you factor in the fact that tokens go stale.

Love to all Beings,

—greg

Hope is beyond reason and Love is even greater than that.

On Sun, Feb 25, 2024 at 13:12 Jim Whitescarver @.***> wrote:

tree of shards

Not sure why you refer to "shards". More clarification may be needed. I am talking about a complete tree in every validator.

It occurs to me that if this is accomplished it might also allow a significantly more effective means for block merge.

— Reply to this email directly, view it on GitHub https://github.com/rchain/rchip-proposals/issues/53#issuecomment-1963062871, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAASW2SWDW3DJBIZMLOBTI3YVOSKNAVCNFSM6AAAAABDRKB3L6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRTGA3DEOBXGE . You are receiving this because you commented.Message ID: @.***>

[jimscarver]

Each shard is a collection of validators serving a given namespace

Thanks for describing decentralization with shards. It's good information. But this thread is not about sharding.

[leithaus]

Dear Jim,

Thanks for your clarification. As i mentioned earlier, there is no transactionally safe decentralization of RSpace below the shard.

Love to all Beings,

—greg

Hope is beyond reason and Love is even greater than that.

On Sun, Feb 25, 2024 at 17:33 Jim Whitescarver @.***> wrote:

Each shard is a collection of validators serving a given namespace

Thanks for describing decentralization with shards. It's good information. But this thread is not about sharding.

— Reply to this email directly, view it on GitHub https://github.com/rchain/rchip-proposals/issues/53#issuecomment-1963101339, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAASW2XM3ARPBUUKJ2YEKQ3YVPC57AVCNFSM6AAAAABDRKB3L6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRTGEYDCMZTHE . You are receiving this because you commented.Message ID: @.***>

[PatrickMockridge]

I spoke with Jim today about this conversation. My conclusion is that Jim hasn't properly defined the question he's asking. After asking @jimscarver to elaborate on the issue a bit more, my suggestion was that he write up a short summary with some diagrams to explain the problem statement and the solutions space in which he's querying.

[jimscarver]

Considering an implementation it may not be practical to implement what I am suggesting.

My thought was that rather than always having the continuation in tuple space it might have a namespace tuple space reference which would activate the referenced namespace to find the continuation and complete it using the union of the existing space and the new space. Each propose would start with only the global tuple space which would not contain most names and continuations. If for example we have huge data blocks like videos their tuple space would only be activated when they are actually accessed in a proposal.

I have not fully considered the potentially huge number of tuple spaces and when to search them in parallel or not. Some heuristic may be needed to effectively decide when a "new" warrants its own tuple space or not such as setting a minimum size or even identifying Objects or Actors in the code. I expect there are more issues. But I am concerned about the explosion of the size of tuple space. Sharding provides a partial answer for now, when employed. This may be an issue for future research. It may or may not have a practical solution witout changing the rho cacl.

On Mon, Feb 26, 2024 at 1:12 PM Patrick Mockridge @.***> wrote:

I spoke with Jim today about this conversation. My conclusion is that Jim hasn't properly defined the question he's asking. After asking @jimscarver https://github.com/jimscarver to elaborate on the issue a bit more, my suggestion was that he write up a short summary with some diagrams to explain the problem statement and the solutions space in which he's querying.

— Reply to this email directly, view it on GitHub https://github.com/rchain/rchip-proposals/issues/53#issuecomment-1964816581, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAXFQELNZSJXLKLUSTXYKULYVTGBZAVCNFSM6AAAAABDRKB3L6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRUHAYTMNJYGE . You are receiving this because you were mentioned.Message ID: @.***>

[leithaus]

Dear Jim,

Thanks for your thoughts and the stimulating discussion.

At the end of the day there is a MUCH faster way to execute rholang. Our experiments are showing 3 orders of magnitude improvement, at least.

Going through Pentti Kanerva’s hypervector formalism is many times faster.

Love to all Beings,

—greg

Hope is beyond reason and Love is even greater than that.

On Mon, Feb 26, 2024 at 17:52 Jim Whitescarver @.***> wrote:

Considering an implementation it may not be practical to implement what I am suggesting.

My thought was that rather than always having the continuation in tuple space it might have a namespace tuple space reference which would activate the referenced namespace to find the continuation and complete it using the union of the existing space and the new space. Each propose would start with only the global tuple space which would not contain most names and continuations. If for example we have huge data blocks like videos their tuple space would only be activated when they are actually accessed in a proposal.

I have not fully considered the potentially huge number of tuple spaces and when to search them in parallel or not. Some heuristic may be needed to effectively decide when a "new" warrants its own tuple space or not such as setting a minimum size or even identifying Objects or Actors in the code. I expect there are more issues. But I am concerned about the explosion of the size of tuple space. Sharding provides a partial answer for now, when employed. This may be an issue for future research. It may or may not have a practical solution witout changing the rho cacl.

On Mon, Feb 26, 2024 at 1:12 PM Patrick Mockridge @.***> wrote:

I spoke with Jim today about this conversation. My conclusion is that Jim hasn't properly defined the question he's asking. After asking @jimscarver https://github.com/jimscarver to elaborate on the issue a bit more, my suggestion was that he write up a short summary with some diagrams to explain the problem statement and the solutions space in which he's querying.

— Reply to this email directly, view it on GitHub < https://github.com/rchain/rchip-proposals/issues/53#issuecomment-1964816581>,

or unsubscribe < https://github.com/notifications/unsubscribe-auth/AAXFQELNZSJXLKLUSTXYKULYVTGBZAVCNFSM6AAAAABDRKB3L6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRUHAYTMNJYGE>

. You are receiving this because you were mentioned.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/rchain/rchip-proposals/issues/53#issuecomment-1965473531, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAASW2RI6NN7VZ56JM2CUKTYVUG4JAVCNFSM6AAAAABDRKB3L6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRVGQ3TGNJTGE . You are receiving this because you commented.Message ID: @.***>