TokTok / c-toxcore

The future of online communications.
https://tox.chat
GNU General Public License v3.0
2.25k stars 284 forks source link

Public unmoderated channels in group chats, found by arbitrary ID (custom name) #581

Open SlugFiller opened 7 years ago

SlugFiller commented 7 years ago

A public channel is similar to a group chat, but differs in how users are managed. In a group chat, users need to be explicitly invited by a moderator of the group chat. In a public channel, subscription is based on a shared secret (arbitrary string), so anyone who knows the secret can join and leave at any time, and no one has special privileges (No way to ban a user).

Also, while a group has an "explicit" existence (one of the clients must create it before inviting others), a public channel has an implicit existence, that is, joining a channel that does not exist creates it.

This could be useful for quickly creating large makeshift groups using a throwaway secret (without the hassle of having to add/verify everyone's ids), or having large persistent public IRC-like channels in cases where moderation is not a priority/requirement.

Having an audio channel which you could freely join/leave (and possibly un/mute specific users) would be even better.

nurupo commented 7 years ago

Public channels? More like secret channels, since you need to know the password to join it, can't join without it.

Good idea, keep the issue open so that we don't forget about it once we get to improving the group chats.

fcore117 commented 7 years ago

I had similar idea too years ago for Tox, would be perfect way to create quick chat like adding temporary Tox id to some website comments(news article for example) and when people click on it then he/she joins a live chat.

iphydf commented 2 years ago

NGC will do this.

SlugFiller commented 2 years ago

I'm not sure how NGC would do this. Group chat requires a person to "open" it and then several people to "join". This suggestion removes the prior requirement - so long as the shared secret is known, it's possible to "join" without anyone having first "opened". Additionally, no one has privileges to approve or deny a "join". I don't see how NGC achieves this.

JFreegman commented 2 years ago

@SlugFiller you can read the spec here: https://github.com/JFreegman/toxcore/blob/ngc_jf/docs/DHT_Group_Chats.md And you can join the testnet and see it working in action by following these instructions: https://gist.github.com/JFreegman/b2bb4d6528ae3e8c8660b72f21ff6f15

SlugFiller commented 2 years ago

@JFreegman From what I'm reading, public groups rely on having a Founder who generates a keypair, in which the public key of said keypair is the chat id. In my suggestion, the "chat id" is not a public key, but rather the hash of an arbitrary string. Aside from not requiring (or allowing) a "founder" who knows the private key matching the public key, it's also a lot easier to share an arbitrary string than a public key, because an arbitrary string can be human-readable, and a public key is not.

JFreegman commented 2 years ago

@JFreegman From what I'm reading, public groups rely on having a Founder who generates a keypair, in which the public key of said keypair is the chat id. In my suggestion, the "chat id" is not a public key, but rather the hash of an arbitrary string. Aside from not requiring (or allowing) a "founder" who knows the private key matching the public key, it's also a lot easier to share an arbitrary string than a public key, because an arbitrary string can be human-readable, and a public key is not.

It would be fairly simple to abstract the Chat ID out using things like public group directories or QR codes. But since all group validation requires certain cryptographic guarantees on the basis of it using a fully decentralized architecture, there's no getting around using keys.

The main issue with using a hash of a string is that multiple people could use the same string, which produces the same hash, and when you join a group there would be no way to verify that you're in the "right" one. If you were to do a uniqueness check on creation, then it would be possible to steal/subvert groups that are currently offline. To solve that problem you would need to use some sort of distributed storage solution. With keys as identifiers none of this is an issue because it's effectively impossible for two peers to create a group with the same Chat ID.

SlugFiller commented 2 years ago

@JFreegman

The main issue with using a hash of a string is that multiple people could use the same string, which produces the same hash, and when you join a group there would be no way to verify that you're in the "right" one

That's the point. The implication is that same string = same group. The idea is that if two people "accidentally" use the same string, then they end up in the same group, even if they made no coordination attempt prior. You prevent it from being the "wrong" group by using a sufficient descriptive string, so that no one would enter the group without having the same idea. For instance, if you create a group from the string "Talks about Mortal Kombat", you can safely assume that if someone else tries to create a group from the exact same string, that they ALSO want to talk about Mortal Kombat, and would be interested in your group anyway.

If you were to do a uniqueness check on creation, then it would be possible to steal/subvert groups that are currently offline

You don't. The whole point of my suggestion is that the groups are not "unique". They are never "created". If person A creates a group with the string "blah", and then person B attempts to create a group with the string "blah", they join the same group as person A, even if they have not coordinate before.

it's effectively impossible for two peers to create a group with the same Chat ID.

And therefore, it defeats the purpose of this suggestion. The whole point of the suggestion is that it's possible for any number of people to "create" the same group, and they are automatically bundled together into a single large group.

JFreegman commented 2 years ago

@SlugFiller I see, it wasn't clear in your initial post that you wanted the lack of uniqueness to be a feature. When you speak of shared secrets it's not automatically assumed that it's okay for random people to accidentally (or intentionally) learn the secret. In crypto-speak a secret has mathematical guarantees to keep it both secret, and unique.

With that said, it's not a bad idea, I'll think about whether or not it's feasible using the changes that we've already made to the DHT in the NGC branch. In the mean time, it's trivial with NGC to create a public group and "disable" moderation by having the founder exit the group after others join it. The only difference would be the Chat ID instead of the human-readable string to join.

SlugFiller commented 2 years ago

@JFreegman It could still be a "secret" in the sense that you'd need to know/guess an exact string that could be of arbitrary length. Although, in practice, it would be difficult to prevent someone from just scanning the DHT, and using whatever hashes are stored there as a "usable password". It's possible to devise protections against it (e.g. requiring each message to be signed by hashing it with the original string), but I'm not sure it's worth the effort.

P.S. In crypto-speak, "shared secret" usually refers to hash pre-images, in that both sides need equal knowledge of the pre-image, and there isn't the public/private asymmetry.

iphydf commented 2 years ago

DHT-based group chats allow this. No A/V is implemented on them yet.

iphydf commented 10 months ago

NGC is merged now.

SlugFiller commented 10 months ago

I explained before that NGC does not fulfill this. They still require someone to administrate the group and approve every new member. And they still offer no way to find a group by name, only by invite. So no, this is not "completed". If there's no intention to add this, close as "not planned" instead.

JFreegman commented 10 months ago

They still require someone to administrate the group and approve every new member.

No they don't. Groups are fully functional without a founder or moderators being present.

And they still offer no way to find a group by name, only by invite.

You don't need an invite to join a groupchat. You just need to know the group's ID, which may be given to you, or listed publicly somewhere.

SlugFiller commented 10 months ago

Groups are fully functional without a founder or moderators being present.

Are you able to join without anyone approving your request? Are you able to join a currently empty group (no users) with just an ID? Are you able to use an arbitrary string for the ID?

You just need to know the group's ID

Repeating the above: Can you use an arbitrary string for the ID? Can you generate one from an arbitrary human-readable (or otherwise) string?

JFreegman commented 10 months ago

Are you able to join without anyone approving your request? Are you able to join a currently empty group (no users) with just an ID? Are you able to use an arbitrary string for the ID?

Yes, yes, and no. The last one would require some sort of permanent data store within the Tox network, and that's a feature that I don't think there are any plans to implement.

iphydf commented 10 months ago

Name -> ID mapping can be done elsewhere. Indeed that's a rather complicated thing to do securely in an untrusted distributed way (blockchains are one way, and they have significant issues). If you have ideas for it, I'm happy to discuss.

SlugFiller commented 10 months ago

The idea suggested here is to simply use some hash function and/or KDF to perform name->key. If the key itself has no particular constraints over it, and may be transient (The first person joining creates the group, and it is implicitly destroyed when the last person leaves), then this is sufficient.

However, from what I can tell, from shallow testing, not every 64-character hex string is a valid group ID. So such a generator would need to adhere to certain constraints, and/or explicitly instantiate the group (instead of implicitly), which defeats the purpose.

iphydf commented 10 months ago

Would you use the KDF to produce the public key for the group? Presumably you'd have to, otherwise people wouldn't be able to join it by name. In that case, how would you derive the secret key from that public key?

SlugFiller commented 10 months ago

An alternative would be to use KDF to derive the seed for both keys. I'm not sure what's the risk factor of the secret key being known. That's why I said it doesn't look like NGC solves this.

A simpler protocol would be to simply derive a DHT key instead, and broadcast presence/bootstrap info on it. From any currently connected set of peers, a few are chosen to perform periodic broadcast and lookup on the DHT (should be no more costly than online presence announcement), so as to ensure that any "islands" on the same key are eventually merged into one group. Any user joining or creating the group (same operation) starts out as an "island" of size 1, and eventually merges with the others.

The key in such a case is not used for encryption, just lookup and broadcast, and is therefore symmetric. If having an arbitrary unverifiable key in the DHT is an issue (I don't see why, but let's assume), the key can always be the hash of a certain pre-hash, and the pre-hash is used for verification. The pre-hash is then the result of the string hash/KDF.

nurupo commented 10 months ago

The idea suggested here is to simply use some hash function and/or KDF to perform name->key. If the key itself has no particular constraints over it, and may be transient (The first person joining creates the group, and it is implicitly destroyed when the last person leaves), then this is sufficient.

However, from what I can tell, from shallow testing, not every 64-character hex string is a valid group ID. So such a generator would need to adhere to certain constraints, and/or explicitly instantiate the group (instead of implicitly), which defeats the purpose.

Group id is a Curve25519 public key, knowing the corresponding secret key for which allows one to be the group Founder - the highest role in the group. It is indeed technically possible to create a group id based on a group name, allowing anyone to re-generate the group id using just the group name string and use that to join the group. One way to do so is to calculate a 32-byte hash(group-name) digest and use it create the group keypair -- the digest would become the secret key, and the public key, which is the group id, is then calculated off the secret key. However, this has the obvious issue that everyone knowing the group name would know the secret key corresponding to the group id, allowing them to claim to be the group founder, compromising group's moderation capabilities. So while technically possible, this will break group chats, so it's not something we can do.


A simpler protocol would be to simply derive a DHT key instead, and broadcast presence/bootstrap info on it.

What is the DHT key derived from, the group name? Just making sure I understand you correctly. In this case having it as a DHT Id, i.e. an arbitrary 32-byte string, is enough, you don't need a cryptographic key pair. (Also your comment on the key being symmetric makes no sense, there is no need for a symmetric encryption key).

From any currently connected set of peers, a few are chosen to perform periodic broadcast and lookup on the DHT (should be no more costly than online presence announcement), so as to ensure that any "islands" on the same key are eventually merged into one group. Any user joining or creating the group (same operation) starts out as an "island" of size 1, and eventually merges with the others.

Ok. So, say I create a group called "Linux". The information on how to join the group is then stored on the DHT on the nodes closest to the DHT Id of hash("Linux"). Say someone else also creates a group called "Linux" and the similar DHT announcement happens. Now, there are two different groups called "Linux", each with different group ids, each having different founders, different set of moderators, sanctioned users, group settings: topic lock, voice state, etc. At some point your solution is supposed to merge these two groups, you call them "islands", into one. How exactly do you propose to do this? Who will end up being the founder of the resulting group, thus owning the merged group?

You could say a group id that is numerically lowest wins the merge, so everyone migrates to that group and its founder keeps being the founder, while the founder and moderators of the group that has lost join as regular users. But this can be gamed -- a malicious user could intentionally generate a group id that has a low numeric value, thus winning the merge and hijacking ownership of someone else's group chat. Pretty much anything you could use to choose which group merges into which can be gamed: group id, number of users in the group, etc. so any script kiddie could easily take over any group chat community on Tox.

There are also some other issues with this solution, e.g. name squatting is another one. How do you prevent popular names from being name-squatted by someone creating new group chats? A group of malicious actors could just name-squat all dictionary words, brand names, etc. It's rather challenging to moderate what data gets stored in a distributed system, each node has to make a decision on its own and it has to work in a distributed fashion, Tox developers don't have some magic certificates that all nodes in the Tox network trust and would allow us to moderate the mappings, nor do we want to ever have any such power, there should be no authority in an untrusted distributed network.


Anyway, as @iphy has said, this is probably better done elsewhere, off the Tox network, as a web API service (e.g. toxme), a blockchain, etc.

SlugFiller commented 10 months ago

However, this has the obvious issue that everyone knowing the group name would know the secret key corresponding to the group id, allowing them to claim to be the group founder, compromising group's moderation capabilities

The point is for the group to have no moderation capabilities - all users are equal. So it might indeed be better to generate a public key for which there is no equivalent private key.

Now, there are two different groups called "Linux", each with different group ids, each having different founders, different set of moderators, sanctioned users, group settings: topic lock, voice state, etc.

No, the implicit groups suggested here have nothing other than users and maybe chat history (although Tox, AFAIK, only keep chat history history, so not even that). No topic (other than the group name itself), no founders, no moderators, no ability to sanction users in any way. This makes merging the "islands" simply a matter of letting each see messages from the others. That is all.

There are also some other issues with this solution, e.g. name squatting is another one

That is not a problem. Since all users are equal, someone "squatting" a name has no more privileges or gains over someone joining later. They are equal. Neither is "founder". Both are regular users. A "squatter" can do nothing to the group someone joining later can't do.

Again, I explained this again and again in this thread, but the point doesn't seem to come through:

Implicit groups are UNMODERATED. All users in it are EQUAL. The name alone makes the group. Anyone who knows the name can join at any time, and has the same permissions as everyone else. "Joining" and "creating" are the exact same operation: There should be no way to tell a person who "created" the group from someone who joined a year later.

nurupo commented 10 months ago

I see. The new group chat implementation in Tox is moderated, so your proposals won't work with it. If we ever decide to add an unmoderated group chat implementation, then this DHT name mechanism could be up on the table. I don't see why would we want to add an unmoderated group implementation though.

SlugFiller commented 10 months ago

There are two possible use cases.

First, if you're getting a bunch of people from a different medium (e.g. game), and simply use some unique string, it's faster to set up, then to create a group and start passing around invitations. Jit.si uses a similar system - The default channels are unmoderated, and simply use a randomly (or custom) generated string as the only key. This is, for example, used by the Godot Foundation. Members of which commented: "If and when someone stirs enough trouble that we need moderation, we'll switch moderated rooms".

Second, is that it's useful as a public forum, like IRC or Matrix. If the point is just to let anyone interested in, then moderation is not the point. In fact, moderation might not even be practical, since a banned user could easily ban-evade by creating a new Tox "account". This is because new joiners don't need to be approved. Even if you could somehow truly block a single user, if the public forum is populated enough, there's no a way one person, or a small group of people, could hope to moderate it.

In such use cases, where moderation is not a priority, and/or not practical, having the extra speed of "Just use this clear-purpose human-readable string" is a much bigger advantage.

If this is still not convincing, then you can close it as "Not planned" (But not as "Completed").

JFreegman commented 10 months ago

@SlugFiller I mentioned how you can have an unmoderated group in a comment that I posted nearly two years ago:

it's trivial with NGC to create a public group and "disable" moderation by having the founder exit the group after others join it.

Moreover, if rather than exiting the group the founder just does a /disconnect he can always /rejoin the group if moderation ever becomes necessary.

SlugFiller commented 10 months ago

@JFreeman Well, that's partially useful. But can you also then associate that group uniquely with a custom-created string so that anyone else joining (or trying to create) the group can do so without any prior communication from any other person (other than using the same string)?

The suggestion here is not just unmoderated groups. It's unmoderated groups that are associated with strings, where any person can join so long as they use the same string. And there's no need to perform a separate operation to create it. From the perspective of a user, a group already "exists" for any given string. Entering a "new" group is indistinguishable from entering an empty group, i.e. one where all previous users have left.

Therefore:

And it gives:

JFreegman commented 10 months ago

But can you also then associate that group uniquely with a custom-created string so that anyone else joining (or trying to create) the group can do so without any prior communication from any other person (other than using the same string)?

I believe that question has already been answered numerous times.

The suggestion here is not just unmoderated groups.

I understand that. I was responding to your comments about unmoderated groups. Since we can already do that, it's not pertinent to the discussion about user generated custom ID's.

SlugFiller commented 10 months ago

Since we can already do that, it's not pertinent to the discussion about user generated custom ID's.

It is pertinent because it influences whether moderation details need to be stored. The big advantage of "unmoderated" is that there's no need to store details on who the moderator is, and what moderation actions occurred.

that question has already been answered numerous times

And the answer has been "There would be no way to persist group details (such as the identity of the moderator)". An answer which is completely irrelevant in light of the suggested groups being unmoderated, and therefore having nothing that needs persistence.

These are not orthogonal details. One enables the other. That's why answering one while ignoring the other is completely missing the point. Repeatedly. Which is why I end up having to repeat myself in a loop.

JFreegman commented 10 months ago

It is pertinent because it influences whether moderation details need to be stored. The big advantage of "unmoderated" is that there's no need to store details on who the moderator is, and what moderation actions occurred.

Why would this be an issue?

And the answer has been "There would be no way to persist group details (such as the identity of the moderator)". An answer which is completely irrelevant in light of the suggested groups being unmoderated, and therefore having nothing that needs persistence.

You've ignored many of the answers you've been given.

SlugFiller commented 10 months ago

Why would this be an issue?

Because if anything about the group needs to be persisted, then it requires a secondary storage to store the name->group mapping. I was even told something to this effect (Including by you):

The last one would require some sort of permanent data store within the Tox network, and that's a feature that I don't think there are any plans to implement.

You just need to know the group's ID, which may be given to you, or listed publicly somewhere.

Name -> ID mapping can be done elsewhere. Indeed that's a rather complicated thing to do securely in an untrusted distributed way (blockchains are one way, and they have significant issues). If you have ideas for it, I'm happy to discuss.

At some point your solution is supposed to merge these two groups, you call them "islands", into one. How exactly do you propose to do this? Who will end up being the founder of the resulting group, thus owning the merged group?

All of the above are issues that stem from the need to persist group information, and the only group information that needs to be persisted is moderation related.

My solution is to have zero-persistence groups (no need for "an untrusted distributed way") by having no "founder" that needs persisting. Name->group mapping is done using a deterministic method (e.g. hash, KDF), and nothing else needs to be added or persisted.

To the extent of that being possible in NGC, I was told (emphasis mine):

One way to do so is to calculate a 32-byte hash(group-name) digest and use it create the group keypair -- the digest would become the secret key, and the public key, which is the group id, is then calculated off the secret key. However, this has the obvious issue that everyone knowing the group name would know the secret key corresponding to the group id, allowing them to claim to be the group founder, compromising group's moderation capabilities.

Which is a fancy way of saying "It won't work with NGC because NGC has built-in moderation features that would break it". This brings us back to this feature not being "completed" at all. And no, I'm not misinterpreting it:

The new group chat implementation in Tox is moderated, so your proposals won't work with it.

You've ignored many of the answers you've been given.

I've quoted several of them above, and explained why they're irrelevant to this proposal. If I missed one, you'll have to be more specific as to which. Because all I've seen is either "This is possible with NGC, other than name->group mapping (i.e. the only part that's important)" or "Name->group mapping is impossible because moderation (which is not relevant to this proposal)", to the point where we looped all the way around to "Groups without moderation are possible in NGC (but without name->group mapping, therefore missing the whole point)"

I don't get it. Is there something about my proposal that I'm not explaining properly? Because I'm trying my best to explain every detail. But somehow the main point seems to be getting ignored every time. Just tell me which part was not clear, and I'll do my best to explain it.

Tha14 commented 10 months ago

@SlugFiller I'm pretty sure that everyone here understands what you're asking(as far as I can tell from the responses). From everything I've read so far you seem to want yet another groupchat implementation which would require a lot of time to be written and tested. It comes down to the fact that we, as contributors, have limited time to spend developing tox and there are other more critical issues we need to tend to. If you're willing to fund the development of such a feature then make it known and people will arrange that with you. Also, I'd suggest getting familiar with how tox works and possibly it's codebase to understand why what you're asking for is not particularly easy to implement.

nurupo commented 10 months ago

I don't get it. Is there something about my proposal that I'm not explaining properly? Because I'm trying my best to explain every detail. But somehow the main point seems to be getting ignored every time. Just tell me which part was not clear, and I'll do my best to explain it.

I understand your proposal and I think it's a neat idea -- just like on IRC, multiple users without a prior communication being able to join a chat group based on some common keyword they independently came up with and end up in the same chat group, all done in distributed fashion right on Tox DHT with minimal computational/bandwidth/storage cost, without querying a web server or a blockchain or some other 3rd party service for the name->group mapping.

Would be cool if this could be done with the current moderated NGC, but I don't see how we can make it work. It could be that I'm just missing some obvious solution. It would work only with unmoderated group chats, but it's something that would need to get implemented as they don't exist yet (and the old group chats are going to be phased out afaik). Perhaps not entirely from scratch, but modify the existing NGC to allow unmoderated group chats. The value of adding unmoderated group chats is kind of dubious though.

Benefits:

Drawbacks:

I don't think that trading moderation for the convenience of joining a group chat is a good trade-off. Now, if we could have that benefit without the group chat being unmoderated, then this would be indeed interesting. If I remember right, we had some GSoC students many years ago working on... was it democratic? group chat -- basically a group chats where moderation is done by voting. It isn't as good as founder-based moderation as it can be gamed, but it is something.

SlugFiller commented 10 months ago

would require a lot of time to be written and tested. It comes down to the fact that we, as contributors, have limited time

I understand that. If you'll notice, this suggestion is 6 years old now, and I'm not rushing or pressuring anyone.

The problem I have is that it's closed as "completed", which implies that this feature already exists in Tox. Which it does not.

If this is simply a matter of "We're busy implementing other stuff, and there's little to no chance of us implementing it", then it could be closed as "not planned". I wouldn't be particularly excited about it, but I'd understand. I wouldn't blame anyone.

But closing it as "completed" gives the false impression that the feature already exists. This could influence future users who might be looking for that feature, but don't find it anywhere. It could influence future requests for this or similar features which might be closed as "duplicate", not by virtue of lack of resources, or failure to match the code base, but rather through a false assumption that the feature is there, but users just can't find it for some reason.

This is precisely why GitHub has "reason" tags when closing issues.

In fact, Tox might even get publicly known as a lawless place worse than 4chan.

Right now, it has the same reputation as early-days Linux. i.e. You need to be at least this much of a wizard to enter. The fact that Toxic (text-based) is the only client that is actively maintained partially contributes to this.

If and when it does hit mainstream, it's going to inherently suffer from the path laid forth by the likes of Telegram. Heck, even the relatively benign Signal has a public impression of some sort of underground. Whenever I say I don't use Whatsapp, I'm asked what I need to hide from the FBI. Basically, that ship has sailed, and it's not coming back.

With that said, while a channel named "Overwatch" might normally attract a lot of unwanted attention, the fact that you need to be using Tox in order to get to it already filters to a much higher degree than Godot or Libera. People not on the technical/civil side are not likely to be using Tox in the first place.

a group chats where moderation is done by voting

I can't envision any permutation of this that isn't vulnerable to a Sybil attack.

Maybe PoW, but that's just "The bigger the machine, the bigger the vote". By contrast, ban evasion would be trivial, and take almost no resources at all. Especially if using Tox over Tor.

nurupo commented 10 months ago

I can't envision any permutation of this that isn't vulnerable to a Sybil attack.

Right, that's what I meant by "it can be gamed".

My memory is very fuzzy on the details, all I remember is someone bringing something like this up on IRC 9-10 years ago, not even sure if this ended up being more than just an idea. The point I tried to make is that maybe there are some other ways that moderation could be done such that it would be compatible with this idea of user-defined group ids/names. Though again, without having the ground truth to anchor against, like the founder in the NGC, such moderation methods are probably going to be flawed in some way.

nurupo commented 10 months ago

(I added a little correction to https://github.com/TokTok/c-toxcore/issues/581#issuecomment-1811047221 -- somehow got mixed up public but password-protected NGC with private NGC. It doesn't really change anything though.)