search

ssbc / ssb2-discussion-forum

not quite tiny, also not quite large
17 stars 1 forks source link

PPPPP: Private groups, attempt 2 #27

Closed staltz closed 11 months ago

staltz commented 11 months ago

Problems with attempt 1

My first design of private groups (symmetric key group with many members) assumed that we could put all members in an account, and then group activity is just an "internal feed" (a feed published by the account, where every msg.data is encrypted with the symmetric key).

The main problem with this is that the account tangle would leak what the members of the group are, and whenever a content msg was published on the internal feed, we would also know what member is publishing.

Requirements

So let's restart this by highlighting what we want.

Notation

CLICK HERE: Short-form for msgs

Whenever I say `data=DATA metadata={ account=A, domain=D } pubkey=PK` it will represent the short form of: ``` { data: DATA, metadata: { dataHash: ..., dataSize: ..., account: A, accountTips: ..., domain: D, tangles: { ... }, v: 3, }, pubkey: PK, sig: ... } ```

Attempt 2

💡 Embed a second "metadata" inside the encrypted msg.data to achieve membership privacy.

💡 We can do better! Let's embed an ENTIRE msg inside msg.data!

Account tangle

Typically, an account works like this: (msg0:) device1_pubkey signs a msg that declares itself in msg.data, then (msg1:) device1_pubkey signs a msg that declares device2_pubkey in msg.data.

What occurred to me is that there could be a "master" pubkey which is shared to all the peers in the account, such that any peer in the account can use that to sign messages. This will gain us membership privacy.

During onboarding a.k.a. Hug :people_hugging: (of a device into the "person" account, OR of a new member into the "group" account), the following information can be exchanged privately, outside of the gossip/replication system:

Here's how the account tangle for the group would look like:

Notice how messages msg1–msg3 are all signed by GROUP_PUBKEY, so outsiders won't know who actually signed this msg (was it Alice? Was it Bob? Was it Carla?) but they can be reassured that it's a valid msg from this account (the group).

:open_hands: Public information:

:shushing_face: Private information:

Internal feed

Now onto the structure of the feed where actual group content is published. This feed is owned by the GROUP account

Notice how msg1 and msg2 are signed by GROUP_PUBKEY, and an outsider cannot know what's going on inside innerMsg, because it is internally encrypted inEnc(innerMsg). And then the innerMsg itself contains more private metadata such as innerMsg.metadata.account and innerMsg.metadata.tangles.

:open_hands: Public:

:shushing_face: Private:

Requirements met?

New problems

:question: What if the inner msg has tangle information too? (In fact it should, because each group member needs to tangle their msgs together) How to construct the inner tangle, what msg IDs do we use for that? How to validate the inner tangle?

:question: For sympathetic replication, we need to allow outsiders to discover the account tangle ID and some internal feed IDs. How should we do that?

:question: How is group exclusion (i.e. epochs) structured? (see ssb-group-exclusion-spec)

mixmix commented 11 months ago

Yeah that last one struck me too (sharing keys across devices - not ideal). nostr all over again?

We need sympathetic replication that

What info is known only to members of a group but noone else - the group secret. Could we obfuscate tangle root data by using that difference? We used this with cloaked msg ids (which were designed to provide a groupId which does not leak the initial message id, and hence author). That alone doesn't get us hidden association with group... unless you make a "per group-member" cloaked id.

Eg I call it group DD2E because that is hash(groupSecret, groupInitMsgId, accountId) And you publicly refer to it by something else because your accountId is different.

This works I think, but don't know what implications are for your replication.

mixmix commented 11 months ago

why can't we do what we did with the "invitations" feed? (expect a bundle of add-member messages you can't open, but when you can it lets you read other add-members, hence discovering all the peers

staltz commented 11 months ago

Yeah that last one struck me too (sharing keys across devices - not ideal). nostr all over again?

This is not a big problem, in my opinion. We are already sharing a symmetric key. We can share an asymmetric keypair, as long as we use that keypair only for a specific purpose: signing internally encrypted msgs only.

gpicron commented 11 months ago

My two cents. When the initial direction becomes too complex, it is sometimes worth resetting and looking in another direction.

Signal has proven to have very efficient and scalable group messaging. The basic principle does not rely on any global secret sharing, which makes it easy to manage membership (which is always a problem with globally shared secrets). All that is needed is a way for actual members to be able to check the membership changes, change their encryption key, and secretly publish it to other actual members.

The sole issue to replicate that is the fact that we do not have a central database holding the 'membership.' That is something that could be resolved using a CRDT OR-SET.

staltz commented 11 months ago

@gpicron Okay, where do see complexity? Sharing of keys isn't complex, in my opinion. The "encrypted msg inside another msg" of this attempt 2 is somewhat complex.

Signal has proven to have very efficient and scalable group messaging.

Signal isn't perfect and it's not a drop-in solution for this case. First, I dislike how Signal groups lose context for new group members, and that's a lost user feature, because we want to include new members and give them access to everything that has been said before. Second, from what I learned in the original notes by Dominic and Keks when they were designing private groups, is that double ratchet is not scalable for large communities. Third, Signal is operating with different assumptions, i.e. a centralized database from which peers can gain consensus. Other solutions like MLS are also designed with a centralized db in mind.

cmoid commented 11 months ago

Third, Signal is operating with different assumptions, i.e. a centralized database from which peers can gain consensus

yea, my thought also when I read this. Signal is very different.

gpicron commented 11 months ago

First, I dislike how Signal groups lose context for new group members, and that's a lost user feature, because we want to include new members and give them access to everything that has been said before

There is a simple solution for that with anchor mechanism that I proposed you and that fit well with PPPP, and each member can choose individual if they want the new member to see or not their previous messages in the group.

Second, from what I learned in the original notes by Dominic and Keks when they were designing private groups, is that double ratchet is not scalable for large communities.

Yes it cannot scale to tens of thousands but it can easily scale to a few thousands. Can the current approach do ? With respect to the in memory db constraint that you impose to ppppp ?

Third, Signal is operating with different assumptions, i.e. a centralized database from which peers can gain consensus. Other solutions like MLS are also designed with a centralized db in mind.

Yes this is the key point. But, they achieved to design a database that is storing zero-knowledge proof of membership and roles making it 'stealable'. A hacker stealing the db or even a Signal admin cannot determine who is part of the a group and who administer a group. So that "centralized membership database" can be in clear visible to everybody. And using CRDT we can perfectly do that using SSB feeds or pppp.

But, forget it. You don't like the idea... You drive. Have a good day

staltz commented 11 months ago

❓ How is group exclusion (i.e. epochs) structured? (see ssb-group-exclusion-spec)

I took the time to try to answer this carefully. @mixmix the following may interest you.

Account tangle only describes epochs

First I think I was wrong about the "account tangle" described in the OP. The account tangle does not need to encode the members of the group, it only needs to encode the "signature keys" for different epochs. So it should look more like this:

graph RL

msg0["declare GROUP_KEY1\nby GROUP_KEY1"]
msg1["declare GROUP_KEY2\nby GROUP_KEY1"]
msg2["declare GROUP_KEY3\nby GROUP_KEY2"]

msg2-->msg1-->msg0

(Recap: GROUP_KEY is used for signing messages published by this group, and for symmetric encryption for group content)

Membership can be expressed by an "external feed"

The group can spawn a new external feed that serves as a CRDT to maintain the set of members. The msgs in this feed can be encrypted to the group AND to the new member (or excluded member). In this way, this feed also serves as the additions feed. Example:

---
title: Members feed
---
graph RL

msg0[account=GROUP,\ndomain=members]
msg1["enc([GROUP_KEY1], {\nadd ALICE by ALICE\n}) by GROUP_KEY1\n\na"]
msg2["enc([GROUP_KEY1,BOB], {\nadd BOB by ALICE\n}) by GROUP_KEY1\n\na,b"]
msg3["enc([GROUP_KEY1,CARLA], {\nadd CARLA by BOB\n}) by GROUP_KEY1\n\na,b,c"]
msg4["enc([GROUP_KEY2,CARLA], {\ndel CARLA by ALICE\n}) by GROUP_KEY2\n\na,b"]
msg5["enc([GROUP_KEY3,CARLA], {\ndel CARLA by BOB\n}) by GROUP_KEY3\n\na,b"]

msg3-->msg2-->msg1-->msg0
msg5 & msg4-->msg3

New comers are "invited" to replicate and decrypt the members feed

If you're going to invite Fred to your group, he doesn't know your Group ID yet. And you don't have an additions feed for Fred to replicate, so how should this work?

  1. You publish an encrypted message (DM, whatever) that Fred can decrypt, saying "I would like to invite you to GROUP_ID, do you consent?"
  2. Fred answers your DM with another DM: "Yes, I consent, go ahead"
  3. Now you publish on the group's members feed, adding Fred and in that msg also sharing the symmetric key
  4. Based on step 1, Fred knows the GROUP_ID, and based on that plus the domain="members" he can replicate that feed
  5. Fred decrypts the msg, and now knows the symmetric key

Or, if Fred doesn't consent, then step 3 never happened, and the only thing Fred learned was the existence of GROUP_ID and its encrypted feeds.

Or, if a lot of time passed between step 2 and step 3, you can choose to not do step 3. In effect, cancelling the invitation (this was an important requirement that Nicholas identified).

Epochs

I'm still going through this, so the following is a big work-in-progress, but I believe we can resolve the 4 different set scenarios (same membership, subset membership, overlapping membership, disjoint membership), but it'll be tailored for these account tangle and members feed structures. Here's an example:

4.4. Same membership

---
title: Members feed
---
graph RL

msg0[account=GROUP,\ndomain=members]
msg1["enc([GROUP_KEY1], {\nadd ALICE by ALICE\n}) by GROUP_KEY1\na"]
msg2["enc([GROUP_KEY1,BOB], {\nadd BOB by ALICE\n}) by GROUP_KEY1\na,b"]
msg3["enc([GROUP_KEY1,CARLA], {\nadd CARLA by BOB\n}) by GROUP_KEY1\na,b,c"]
msg4["enc([GROUP_KEY2,CARLA], {\ndel CARLA by ALICE\n}) by GROUP_KEY2\na,b"]
msg5["enc([GROUP_KEY3,CARLA], {\ndel CARLA by BOB\n}) by GROUP_KEY3\na,b"]

msg3-->msg2-->msg1-->msg0
msg5 & msg4-->msg3
---
title: Account tangle
---
graph RL

msg0["declare GROUP_KEY1\nby GROUP_KEY1"]
msg1["declare GROUP_KEY2\nby GROUP_KEY1"]
msg2["declare GROUP_KEY3\nby GROUP_KEY1"]
msg3["redeclare GROUP_KEY2\nby GROUP_KEY1"]

msg1--"a excludes c"-->msg0
msg2--"b excludes c"-->msg0
msg3-->msg1 & msg2
winner["wins (lexicographically)"]-.->msg1

style winner fill:#0000,stroke-width:0
mixmix commented 11 months ago

@staltz corrections:

mixmix commented 11 months ago

I'll come back to the diagrams and tegh stuff when I'm not at noisy kids gymnastics.

But I think we should consider @gpicron 's suggestions about MLS. Those are strong and valuable claims. Even if we don't use it sounds like there is valuable prior art to check.

Reminder that the envelope spec and tribes came out at about the same time as MLS... so we didn't integrate ideas from that project in owr thinking so far.

@staltz I feel tension with @gpicron here. My take is that they don't feel heard, or what they're sying is seen as irrelevan? (I could be wrong) I notice I'm a bit annoyed by your innacurate history (you forgot I was there and helped design this, or didn't know? And you got details wrong ... I think). I don't care about that that much (we have a good relationship and I know you update your models on feedback). What I do care about is "how do you want ppppp to grow and be governed?" I want it to have a few strong and committed parents guiding and growing it into maturity. This is not a calling out, it's a calling in (death by cheesy) - to invite us to consider, "can we align for this to be a shared project". The alternative is "this is staltz's baby and we are just advisors". That is totally valid too. Maybe I'm just asking to clarify that. (with love :heart:)

derhuerst commented 11 months ago

(I'm just jumping in here making random comments, with the intention of making discussions like this one more discoverable. If my comment is out of scope, please just hide it as such.)

[…] Last I checked the only problem I surfaced was that the [MLS] spec does not have a distributed key cycling process for new epochs (yet). […]

related ticket documenting this problem: https://github.com/mlswg/mls-federation/issues/6

staltz commented 11 months ago

I want to get the conversation back on track, which is "how to accommodate SSB's private message model to PPPPP". Let's have other conversations to discuss other issues, especially the higher-level ones like "how do you want ppppp to grow and be governed?". Not in this issue.

I am ruling out Double Ratchet and MLS and anything else like that simply because it is a research effort (distributed systems and cryptography is no picnic, and it usually takes real academic effort) and/or implementation effort for which we have no resources yet. Hopefully one day we'll have the resources (time, money, team, reviewers) to do all that properly. What we have resources for now is to port existing SSB mechanisms which – despite crippled – are implemented and are proven to work well enough between users. As a metaphor, PPPPP is a new house, and we're just moving furniture from the SSB house to this one, and we need to see how to place the furniture in the best places. We are NOT buying new furniture, and we're definitely not building furniture ourselves. So can we focus on how to move furniture? Just picking things up and putting them where they make sense.

@mixmix with all that said, I want to see what you think about those diagrams and since you're one of the few with loaded context on private groups and metafeeds, whether it seems like the "exclusion spec" furniture could fit in this new "accounts and tangles" bedroom.

staltz commented 11 months ago

More technical comments:

Encryption scheme: Given the flexibility of envelope-spec to have all sorts of recipients, and that both ssb-box1 (private-box) and ssb-box2 (envelope-spec) are based on the same libsodium.crypto_secretbox_easy primitive, I think there's no point in doing private-box, we should just use envelope-spec for all purposes. This will simplify code too. However, I think I'll have to adapt some minor details of envelope-spec e.g. it assumes feed_id and prev_msg_id are available information. Works for SSB but not for PPPPP.

Validation complexity: The idea of embedding a {data,metadata,pubkey,sig} msg inside a msg.data seems like it will work well, but it's going to make implementation of the DB module a lot more complex. This is also the only unanswered question so far from the list of open questions I posted in the OP. Recap:

❓ What if the inner msg has tangle information too? (In fact it should, because each group member needs to tangle their msgs together, to assert order of their msgs) How to construct the inner tangle, what msg IDs do we use for that? How to validate the inner tangle?

Key management: I'm not sure if there should be a symmetric key for internal communication. What if there is only a "internal keypair" which is used for both signing the outer msg and encrypting/decrypting the inner msg (once the ed25519 keypair is converted to curve25519)?

staltz commented 11 months ago

Here's an early attempt at specifying the outer msg (containing the inner msg in data):

{
  data: encrypted([GROUP_KEY, BOB], {
    data: {
      // ... add Bob to the group
    },
    metadata: {
      account: ALICE,
      // accountTips,
      // dataHash,
      // dataSize,
      domain: "members",
      tangles: {
        [ALICE_MEMBERS_FEED_ROOT]: {
          // ...
          // msg IDs here are inner msg IDs
        }
      }
    },
    pubkey: ALICE_DESKTOP,
    // sig,
  })
  metadata: {
    account: GROUP_ID,
    // accountTips,
    // dataHash,
    // dataSize,
    domain: "members",
    tangles: {
      [MEMBERS_FEED_ROOT]: {
        // ...
        // msg IDs here are outer msg IDs
      }
    },
    v: 3,
  },
  pubkey: GROUP_KEY,
  // sig,
}
ahdinosaur commented 11 months ago

Key management: I'm not sure if there should be a symmetric key for internal communication. What if there is only a "internal keypair" which is used for both signing the outer msg and encrypting/decrypting the inner msg (once the ed25519 keypair is converted to curve25519)?

would there be a performance impact to using an asymmetric keypair for encryption and decryption? my intuitive understanding is that asymmetric keypairs are less performant, hence why are usually used to bootstrap a symmetric key. i understand the appeal to having less keys to manage, but at the same time i think why use an asymmetric key pair for encryption / decryption when all parties have both sides of the key pair? also as an aside, i don't think this would have an impact due to the signing, but it also means anyone (who isn't a member) could encrypt a message using the public key (but they wouldn't be able to sign such a message using the private key).

staltz commented 11 months ago

@ahdinosaur That's a strong point, thanks for the reminder! And yeah, besides the performance and misuse by strangers, perhaps it's better to have the asymmetric keypair only for signing, and the symmetric key only for encryption. Case closed

mixmix commented 11 months ago

Thoughts:

1. you can't use GROUP_KEY in the two ways you've described

I think you mean

2. how does Bob decrypt this?

Currently, when "bob" is included in the recps, that means "make a shared key by crossing the authors SK with bob's PK". Then when bob receives this he knows he can try a shared key (which he makes from his SK and the authors PK). Buuutt, who is the author here? msg.metadata.account = GROUP_ID ... this then requires an encryption key for the group ... is that GROUP_PK (ed25519) converted into GROUP_PK (curve25519)?

As you know, I like the idea of being able to cycle keys... it may be a requirement for groups which support exclusion even?

staltz commented 11 months ago

@mixmix Yes, you're right in those details.

A group should declare:

As you know, I like the idea of being able to cycle keys... it may be a requirement for groups which support exclusion even?

Yes and yes.

staltz commented 11 months ago

I found a problem that could be a deal breaker.

With the wrapping idea, it seems like we cannot support deletion of the inner msg.data and (thus) partial replication. It seems like deleting the inner msg.data will change the ciphertext, and thus change the outer msg.metadata.dataHash and thus the outer msg ID and sig. Partial replication, i.e. sliced replication of a minDepth / maxDepth range of the tangle is a huge requirement for PPPPP and if groups cannot support it, we'll be in trouble quickly, regarding storage usage.

The only other alternative I have in mind is leaving the author's information on the encrypted msg, i.e. group content msgs would look like this:

{
  data: encrypted([GROUP_KEY, BOB], {
    // ... add Bob to the group
  }),
  metadata: {
    account: ALICE,
    // accountTips,
    // dataHash,
    // dataSize,
    domain: hash(GROUP_ID+"members"),
    tangles: {
      [ALICE_MEMBERS_FEED_ROOT]: {
        // ...
        // msg IDs here are inner msg IDs
      }
    }
  },
  pubkey: ALICE_DESKTOP,
  // sig,
}

So whoever (group outsider) gets a hold of this msg will know that it belongs to Alice, but they don't know which group it belongs to, since the GROUP_ID is hashed. Sympathy replication would still work because given account=ALICE & domain=hash(GROUP_ID+"members") you would know the moot ID and thus can fetch the whole feed for Alice's additions of members to this group, without knowing (as a group outsider) what these msgs are actually about.

The downside is that we can't do a "pruning CRDT" (see thesis) of the membership set, because there isn't a single feed for that, there is Alice's membership-mutation msgs, Bob's membership-mutation msgs, etc. In each of these feeds we can do pruning, but the opportunities for that are rare, since we can't "refactor" the membership set like we can if we had one feed.

So in essence, partial replication here is also damaged.

staltz commented 11 months ago

Oh, wait!

I think the wrapping idea could still work with partial replication, we just need to delete the outer msg.data, then the outer msg.metadata is preserved etc.

Regarding tangles in the inner msg, I'm considering leaving it totally empty. Basically the inner msg would be "orphan", it doesn't belong to anything.

It makes sense when you think of the ultimate purpose of msg.metadata.tangles, which is to assert what msgs came after what, an ordering guarantee. If we already have an ordering guarantee from the outer msg tangle, we don't need a second ordering guarantee. What the outer msg tangle does is provide or ordering guarantee across many accounts participating in the group, while the inner msg provides ordering guarantee for a single account. Seems like the outer tangle is stronger and we don't need the inner one.

staltz commented 11 months ago

Alright, I think I settled on a design, and for reference I'll (re)describe it in this comment, to centralize the ideas.

Account tangle only describes epochs

A group account needs three types of keys:

The account tangle has:

Example:

graph RL

msg0["GROUP_SIGKEY1\ndeclares\nGROUP_SIGKEY1"]
msg1["GROUP_SIGKEY1\ndeclares\nGROUP_EKEY1"]
msg2["new epoch!\n\nGROUP_SIGKEY1\ndeclares\nGROUP_SIGKEY2"]
msg3["GROUP_SIGKEY2\ndeclares\nGROUP_EKEY2"]
msg4["..."]

msg4-->msg3-->msg2-->msg1-->msg0

Members feed

Maintains the set of members. The account=GROUP_ID (msg ID of the root of the account tangle) and domain="members". The msgs in this feed are encrypted (symmetrically) to the group and (asymmetrically, Diffie-Hellman) to the new member (or excluded member).

The msg.data is yet another msg, called the "inner msg", where innerMsg.pubkey and innerMsg.metadata.account refer to a specific member of the group. Contrast that with the (outer) msg.pubkey which is GROUP_SIGKEY and msg.metadata.account which is GROUP_ID. The inner msg does not need to have tangles, i.e. innerMsg.metadata.tangles = {}, but the outer msg needs to be tangled as the members feed.

For partial replication, the inner msg can be deleted entirely, leaving the outer msg.data to be null.

Example:

graph RL

msg0[moot\n\naccount=GROUP_ID,\ndomain=members]
msg1["GROUP_SIGKEY1\npublishes\nenc([GROUP_IKEY1], {\nALICE adds ALICE\n})\n\na"]
msg2["GROUP_SIGKEY1\npublishes\nenc([GROUP_IKEY1,BOB], {\nALICE adds BOB\n})\n\na,b"]
msg3["GROUP_SIGKEY1\npublishes\nenc([GROUP_IKEY1,CARLA], {\nBOB adds CARLA\n})\n\na,b,c"]
msg4["GROUP_SIGKEY2\npublishes\nenc([GROUP_IKEY2,CARLA], {\nALICE excludes CARLA\n})\n\na,b"]
msg5["GROUP_SIGKEY3\npublishes\nenc([GROUP_IKEY3,CARLA], {\nBOB excludes CARLA\n})\n\na,b"]

msg3-->msg2-->msg1-->msg0
msg5 & msg4-->msg3

Group onboarding

New comers are "invited" to replicate and decrypt the members feed.

  1. You publish an encrypted message (a DM) that Fred can decrypt, saying "I would like to invite you to GROUP_ID, do you consent?"
  2. Fred answers your DM with another DM: "Yes, I consent, go ahead"
  3. You publish a msg on the group's members feed, adding Fred and in that encrypted msg also informing GROUP_IKEY (secret), GROUP_SIGKEY (private) and GROUP_EKEY (private)
  4. Based on step 1, Fred knows the GROUP_ID, and based on that plus the domain="members" he can replicate the members feed
  5. Fred decrypts the msg, and now knows the keys to the group

Or, if Fred doesn't consent, then step 3 never happened, and the only thing Fred learned was the existence of GROUP_ID and its encrypted feeds.

Or, if a lot of time passed between step 2 and step 3, you can choose to not do step 3. In effect, cancelling the invitation (this was an important requirement that Nonlinear identified).

Epochs

I'm still going through this, so the following is a big work-in-progress, but I believe we can resolve the 4 different set scenarios (same membership, subset membership, overlapping membership, disjoint membership), but it'll be tailored for these account tangle and members feed structures. Here's an example:

Same membership

---
title: Members feed
---
graph RL

msg0[moot\n\naccount=GROUP_ID,\ndomain=members]
msg1["GROUP_SIGKEY1\npublishes\nenc([GROUP_IKEY1], {\nALICE adds ALICE\n})\n\na"]
msg2["GROUP_SIGKEY1\npublishes\nenc([GROUP_IKEY1,BOB], {\nALICE adds BOB\n})\n\na,b"]
msg3["GROUP_SIGKEY1\npublishes\nenc([GROUP_IKEY1,CARLA], {\nBOB adds CARLA\n})\n\na,b,c"]
msg4["GROUP_SIGKEY2\npublishes\nenc([GROUP_IKEY2,CARLA], {\nALICE excludes CARLA\n})\n\na,b"]
msg5["GROUP_SIGKEY3\npublishes\nenc([GROUP_IKEY3,CARLA], {\nBOB excludes CARLA\n})\n\na,b"]

msg3-->msg2-->msg1-->msg0
msg5 & msg4-->msg3
---
title: Account tangle
---
graph RL

msg0["GROUP_SIGKEY1\ndeclares\nGROUP_SIGKEY1"]
msg1["GROUP_SIGKEY1\ndeclares\nGROUP_EKEY1"]
msg2["GROUP_SIGKEY1\ndeclares\nGROUP_SIGKEY2"]
msg3["GROUP_SIGKEY1\ndeclares\nGROUP_SIGKEY3"]
msg4["GROUP_SIGKEY1\nredeclares\nGROUP_SIGKEY2\nand deletes\nGROUP_SIGKEY3"]

msg1-->msg0
msg2--"a excludes c"-->msg1
msg3--"b excludes c"-->msg1
winner["wins (lexicographically)"]-.->msg2
msg4-->msg2
msg4-->msg3

style winner fill:#0000,stroke-width:0

For figuring out which epoch to prefer, we need the group membership set, and at a certain state, so it seems like we will need to know the tips of the members feed declared in each account msg, with the exception of the first account msg.

msgOnAccountTangle = {
  data: {
    // ... add a new SIGKEY
    membersTips: [/* ... */], // msg IDs of the current tips of the members feed
  },
  metadata,
  pubkey,
  sig,
}
ahdinosaur commented 11 months ago

well done @staltz, very exciting :relaxed:

With the wrapping idea, it seems like we cannot support deletion of the inner msg.data and (thus) partial replication. It seems like deleting the inner msg.data will change the ciphertext, and thus change the outer msg.metadata.dataHash and thus the outer msg ID and sig. Partial replication, i.e. sliced replication of a minDepth / maxDepth range of the tangle is a huge requirement for PPPPP and if groups cannot support it, we'll be in trouble quickly, regarding storage usage.

with Zen of Python "flat is better than nested" in mind, could an alternative approach be to not nest the encrypted data and metadata. so data is still data, now encrypted, and groupMetadata (with the group's inner metadata, pubkey, and sig), also encrypted.

staltz commented 11 months ago

For completion, here's how private messages (between e.g. Alice and Bob) would work:

Private messages

Encryption keys:

Account tangle

Onboarding hugs

With new devices or new apps:

External feed for private messaging

Internal feed for intra-account communication

staltz commented 11 months ago

with Zen of Python "flat is better than nested" in mind, could an alternative approach be to not nest the encrypted data and metadata. so data is still data, now encrypted, and groupMetadata (with the group's inner metadata, pubkey, and sig), also encrypted.

@ahdinosaur I'm not sure if that can be done without making msgs have a dynamic shape, which is also kind of bad. I'm trying to go for "static shapes are better than dynamic shapes" and currently the shape we have is:

{ data, metadata, pubkey, sig },

and we only have freedom inside data.

staltz commented 11 months ago

The discussion can continue, but I want to signal that I found a solution. Closing!