Open srkunze opened 9 years ago
also, as I said, people would expect to get correct contacts back. Skype does not verify your identity either afaik, but at least requires you to have an e-mail account, no?
Updated.
Aren't we talking about authentication again? I guess that need to be addressed by those 3 or 4 different approaches we talked about in that other thread.
that is for security aware people. but 'normal' people? I'm a bit torn here. Skype does not verify afaik, but likely limits account creation per e-mail address. or do you not need one? whatsapp otoh authenticates its users as having access to the number...
I don't know what you mean. Aren't those approaches discussed elsewhere enough protective measures? Normal people will just do whatever the authentication protocol requires them to do. So, clients should make it a no-brainer, right?
Otherwise, how do you plan to deploy such verification procedure?
You can create a fake account on Skype using a fake email at least, or even a legit one if you have sufficient control over the provider (#NSA). For WhatsApp, same last issue: I think gov agencies can access your phone messages.
The goal is to bring security to everyone, not only aware people. That require telling people when they add a contact how to authenticate it securely. Le 1 févr. 2015 20:38, "aaannndddyyy" notifications@github.com a écrit :
that is for security aware people. but 'normal' people? I'm a bit torn here. Skype does not verify afaik, but likely limits account creation per e-mail address. or do you not need one? whatsapp otoh authenticates its users as having access to the number...
— Reply to this email directly or view it on GitHub https://github.com/irungentoo/toxcore/issues/1222#issuecomment-72380108.
@aaannndddyyy , you are right that you don't need to check the DHT every time for contact searches, only when a user decides to do a search. Though someone that wants to be found based on say phone number, does have to announce it to the DHT. This has to be periodically, otherwise the nodes will drop this phone-book entry, or may simply go offline. If nodes wouldn't drop it after a certain time it would be easy to render the DHT useless by filling it with large amounts of garbage. The periodic storage requests would create additional traffic.
@srkunze , the multiple sets of hashed keys is not for security but simply to be able to be found on e.g.: ToxID, family name, e-mail, and phone number. That is already four entries instead of just a ToxID today. Hashing is not for security, just to make it the same number as bits as a DHT key. Storing the real data would allow partial matches, though I think that this would be hard to implement without straining performance. Besides, Skype doesn't do it and is pretty user friendly.
As I proposed 6 days ago in this thread, it would be possible to allow fast searches on e.g. first name, last name, city, as well as all combinations thereof. I think this is crucial because some names are just too common. It doesn't matter much if we'd pick the fields people can use or not, it can be free text for those potentially non-unique fields. The only distinction I would make is non-unique (names, city, etc. ) versus unique fields (e-mail, phone). Not specifically stating which non-unique fields would allow a user to type "Tom Smith" or "Smith Edinburgh" in a search box without the client having to guess what is a name and what is a city. All combinations of the non-unique fields should be hashed, while the unique fields should only be hashed individually. The unique fields can also be standardized before hashing (e.g. phone numbers), so no free text here. In sum, I prefer 'free text' for non-unique identifiers such as names and cities, at least in the background, because I think it is easier for the user and there is no downside to it. Clients could of course still show input boxes for first/last/city and other optional info as a guide to users that want to announce their ToxID to the world.
Something else I was thinking. Why not have a bot like toxme.se where clients could communicate with in a simple protocol language? Clients could push a button to publish to toxme.se without having to set up an account. Authentication would be automatic with the public/private key pair. Users could update the info linked to their ToxID/public key. This would be much simpler, though not truly distributed any more.
All combinations? I publish my id under Tom Smith Edinburgh. And Alice is searching for Tom Smith, without any city. Alice should find me. I had thought about having at max five records:
Maybe even only four and have the optional location in the clear and let the client sort or filter by location. This way, if the searcher does not know the exact location, he can still look at the ones returned and see which record is the most likely match.
Perhaps it would be best to just have location hashed on its own, that way they could search for just first name and last name, as well as location, or email and location, more combinations. Also, I think it would be nice if there was a way to display avatars for search results, as that makes: "John Smith" and "John Smith" easier to tell apart. Well which John Smith is mine, oh yeah it's the moron with a cat for an avatar. I know I personal at least recognize my friends on Skype by their avatars, since they change their name so often. If they change both avatar and name then we have to go through the awkward ordeal of "Who are you, why did I add you, ok now prove it". Sadly displaying the avatars would require more storage space, or a connection to the user.
On Sunday, February 1, 2015, aaannndddyyy notifications@github.com wrote:
All combinations? I publish my id under Tom Smith Edinburgh. And Alice is searching for Tom Smith, without any city. Alice should find me. I had thought about having at max five records:
- phone number -e-mail
- user name
- first name, last name
- first name, last name, location
Maybe even only four and have the optional location in the clear and let the client sort or filter by location. This way, if the searcher does not know the exact location, he can still look at the ones returned and see which record is the most likely match.
— Reply to this email directly or view it on GitHub https://github.com/irungentoo/toxcore/issues/1222#issuecomment-72405462.
Sadly displaying the avatars would require more storage space, or a connection to the user.
Nothing is for free. ;)
@Dirius77 , a connection to the user should be no problem for a distributed communication system as Tox, right? Avatar and additional info could be queried automatically to the search results. I'd prefer that.
I don't see how storing a DHT entry for just the location would be helpful. Once Tox goes mainstream this is bound to return too many results to handle. If not all results are returned, filtering by the searching client may not turn up the contact we are looking for. In some cases the same may be true for names. I can imagine somebody looking through a list of all John Smiths, though this may not be possible with common Chinese names. I am just speculating, though I can imagine that a full name may not always cut it down to tens of results.
What is wrong with storing "Thomas", "Smith", "Tom", "London", and all 11 (sorted) combinations thereof? It would be relatively easy to detect the too common cases by querying the DHT first, and not bothering storing those. I expect that around 15 hash-ToxID pairs should be sufficient to be able to locate users based on phone, e-mail, and any combination of 4-5 other tokens such as first, last, other name, and city. Note that the order doesn't matter, one could always sort alphabetically before hashing a DHT key. I think this would be both user friendly and technically feasible. Though it may require some toxcore changes to handle well.
There are 24 ways to sort your example above. And you mention 15 hash-id records, wheras I envision 4 or five. And that only if user has specified all of his contact info.
@aaannndddyyy , search terms can be sorted in a predefined way (say alphabetically), so in this case only 2^4-1 sets are possible, of which the 4 singletons can probably be removed due to high occurrence, so 11. Granted, that is still more than 4-5, though with the 11 key-ToxID pairs stored the searches would return pre-filtered results only and would still work if you are unsure about for example your contacts first name (say Tom/Thomas, José/Jose/Pepe).
I am not saying that short names should be a publish/search term, this is only useful in some cases. Additional 'free text' terms would be useful though, so it can adapt to whatever people use to find each other in each culture. Moreover, such terms could also be used to filter professions or company names. The possibilities are endless, so personally I would leave pre-defining fields to the clients.
Perhaps the username should remain as the one unhashed value, that way the user would be able to search for a first name or whatever, and recognize the user name when it shows up. I feel as if the chances of two "John Smith"s using the same user name is much lower than them say, living in the same location. And having a lot of results doesn't seem to bother Skype, it will happily show you a list of 800 people and leave it up to you to guess which is which.
On Monday, February 2, 2015, tttom notifications@github.com wrote:
@aaannndddyyy https://github.com/aaannndddyyy , search terms can be sorted in a predefined way (say alphabetically), so in this case only 2^4-1 sets are possible, of which the 4 singletons can probably be removed due to high occurrence, so 11. Granted, that is still more than 4-5, though with the 11 key-ToxID pairs stored the searches would return pre-filtered results only and would still work if you are unsure about for example your contacts first name (say Tom/Thomas, José/Jose/Pepe).
I am not saying that short names should be a publish/search term, this is only useful in some cases. Additional 'free text' terms would be useful though, so it can adapt to whatever people use to find each other in each culture. Moreover, such terms could also be used to filter professions or company names. The possibilities are endless, so personally I would leave pre-defining fields to the clients.
— Reply to this email directly or view it on GitHub https://github.com/irungentoo/toxcore/issues/1222#issuecomment-72451141.
Once Tox goes mainstream this is bound to return too many results to handle. If not all results are returned, filtering by the searching client may not turn up the contact we are looking for.
Not a problem at all. Facebook has the same "problem". To me that is not a problem at all. You refer to location but this can happen to any other search parameter. Just cap a search to 20 result items, and we are fine.
Users will work around it be searching differently and use a more restrictive query if necessary. This is part of Information Retrieval and we do that each day when using Google.
I don't think a centralized list (or limited centralized lists) is the correct way to go. When the whole setup so far is decentralized, creating a centralized tox id -> user@domain list isn't a great idea. It could also be hacked, be hit be ddos, monitored, censored or used to spam What about thousands of user created group-limited servers?
Here's my ideal setup I have two friends (Alice and Bob) I would run a small server (hopefully as easy as ./tox_user_server -p 2500 -f userlist) that basically contains three id's. My domain name is lalatox.org and ip is 4334.345.3.3 (domain being optional) Alice HJKH245K Bob KJHJJHKL Mecvak IUYOO2U354
Alice would add Bob by either adding Bob@lalatox.org, Bob@4334.345.3.3, or KJHJJHKL. Since I own the server there are a few advantages
This is also not limited to just the two bits of info. More info like contact info could be added (only accessible once the friend accepts the invite / enters correct password)
This could be scaled to centralized lists (like Facebook) but it should be considered a secondary "global" search and not the main option.
Couldn't you use the Kontalk method? "Kontalk uses your phone number to identify yourself and automatically adds other Kontalk users you can talk with by looking in your contact list. [...] Even in server-to-client communication, your phone number is irreversibly encrypted, so even the server can't know your phone number: it is used only for sending you the verification code, then it will be discarded." (https://f-droid.org/repository/browse/?fdid=org.kontalk)
Edit: I just tried Threema and I think they actually have the best approach (as mentioned above): A unique ID is created to identify a user. Optionally, users can link their phone number and/or e-mail address to this ID if they are able to verify them. Furthermore, Threema can optionally scan the user's contact list to automatically add other users that have attached their phone number/mail address to their account. So, you can have the convenience and verification of Whatsapp if you want (which probably most users do), but it is opt-in and you could use the app with manually managing contacts as well.
Anyway, I'm 100% sure that Tox has absolutely no chance of ever reaching a critical mass if there isn't a least an option to automatically handle contacts by phone number. Users just aren't willing anymore to exchange user names and remember a password for IM. This was OK in pre-mobile times, but Whatapp changed the game here and any other IM has to offer the same level of convenience if it wants to reach non-privacy-focused users as well.
@Mecvak I think your idea of decentralized servers could be used additionally to the discovery feature. But it would not work alone for the average user because it's already too complicated to use. It would still make a pretty good alternative for those, who don't want to use the public opt-in function. Like you mentioned it: small organized groups would be able to use such servers. But my grandma already has problems hitting the right letters on her smartphone. Setting up and signing into such a 3rd party server would be too much for the average user.
@Mosrite This sounds rather interesting for the phone number problem. But not all clients are mobile so we may want to use profiles with usernames and [optional: avatar, full name, location, public message]. I think the question is: how do we distribute a profile to make it findable in a decentralized network.
Just to sum up this long thread and what a discovery feature would mean: Like @srkunze mentioned first, we need to find users (if they want to be found ofc - I don't want to repeat that much.. @ArchangeGabriel ) by their
Also this feature should be as decentralized as possible, so that we don't have to trust and rely on 3rd parties. And of course opt-in / toggleable - you decide when someone is able to find you over the network. (Do you need to be opt-in to use this feature at all?)
So here an example of how I imagine user discovery (with profiles):
(without public profiles)
With profiles is more userfriendly but might be harder to implement. Also you just exchange ToxIDs when you authenticated each other and don't present it connected to your username in public.
We have (so far) 2 options on how the query could look like:
Both options have their pros and cons. Which one would be practicable, I think, depends on where the information we are looking for is stored:
(I might miss some pros or cons, so if you know some important ones, please mention them) Now we have send our query and we have to take a look at how does the results look like :
The last thing is authentication (Greatly explained in #1271 ) In the end we want to have a correct ToxID added to our friendlist.
This was the general summary of the (so far mentioned) options and their pros and cons.
Now I want present you my idea. Tell me if this would be even possible and/or useful.
¹ we could first try the authentication here and exchange ToxIDs after successful authentication
This version really wants to exchange the ToxIDs as the last act. It might be easier to use them earlier so the DHTs won't have that much tokens to handle. Optionally the DHTs could cache which clients answered on which queries for a given amount of time. Also a client may only query once at a time and DHTs may be able to [temporary] block certain spamming clients. DHT's might be opt-in/out too, if they are not able to handle the overhead and work just as they work now. Tokens should completely handled internally by DHTs. Queries could still be hashed or plain text.
Hopefully you understood everything what I meant :x Maybe I should create some diagrams xD
How about some last pros and cons?
Finally a huge thanks just for reading this giant wall of text. That was not even intended originally :x
Sorry for being that destructive but I'm pretty sure that none of your scenarios (with profiles or without) will have any chance to get a critical mass of users. If you just want a very secure IM for privacy-focused users only, that's fine. But if you want to create a secure messenger for everybody the approach must be way easier. Most people just want to install an app, verify their phone number and that's it. If they have to come up with a username and a password and have to manually query for contacts they just feel that this is way too inconvenient and won't install the app in the first place.
Of course, there should be an option (if opt-in or opt-out doesn't really matter eventually) to use the app with an ID or even a self chosen username and to manage contacts manually. And you're right, you cannot verify a phone number on a desktop machine. But messaging takes place mostly mobile these days. Most desktop users have a smartphone as well and can verify their computer with their smartphone (e.g. Telegram works that way). And additionally, they could just not add a verified phone number to their ID (but managing contantcs automatically wouldn't be possible anymore). So, the desktop thing is not really an important issue I think. Just look at Whatsapp: It wasn't usable from a desktop computer until some weeks ago at all, and now there are only browser implementantions that don't even work pretty well. But noone cares! Most non-tech-savvy users don't even think cross-plattform, they just want a mobile messaging app that works, that is extremly easy to handle and that offers a critical mass of contacts.
My suggestion was primarily focused on tox-core here. How the clients (e.g. mobile) implement and handle these base features is their responsibility. Even if my description was written with desktop in mind (as I see tox as a desktop skype alternative and less as a mobile IM) and I mentioned that a user initiates a search - clients could still implement automated lookups. Let's take Antox for example: You install it on your phone and it automatically queries the network with your contactsbook (with your permission of course). Because phone numbers should be unique there shouldn't be much results. If for whatever reason there are more results te client could match the retrieved profiles with your local contactinfo. If there are still more than one results, well then the user can be asked to choose.
This was just a quick example. How the clients use these library functions is their permission and doesn't need to be specified here in tox-core in my opinion.
Yes, in the perpective of user-friendliness my idea will not work well. Even in a one-click solution there would be downtimes as users generally don't run computers on 24/7 (although this is becoming more frequent - especially in mobile) Perhaps "pushing" of association of aliases / tox id's to nodes to temporaily store.
More simply:
This has a few pro's and con's.
P
as I see tox as a desktop skype alternative and less as a mobile IM
OK, if you really see Tox this way phone number based identification might not be that crucial. But (sorry for being distructive again) this brings up two questions again:
I'm neither a programmer nor a cryptgraphy expert, I'm just a user, so I cannot be very constructive, for which I'm sorry. I'm just saying: If you just want to create something that is secure for privacy-focused users and which doesn't focus on usability, that's fine, but there are pretty good alternatives out there, which would make Tox basically redundant. The only reason I could think of for developing something as Tox would be to create something way more user friendly, so that in the end we would have an open source, distributed and secure messenger (text + call + video) which even my grandmother could use (if she had a smartphone or a computer... ;) ).
@Mosrite I don't think you are destructive at all. You made very good points and I could not agree more with you.
@Mosrite well the current situation is: we have security focused desktop applikations (you named them) and we have security focused mobile apps. Both have the following problems:
Tox could combine everything:
That's what I understood as the goal of Tox (but like you, I am no developer here [yet?]). The multiplatform thing is possible because the needed base functions are seperated into the core (this repository here). Every platfrom that wants to tox needs a client which combines the core funtions with platform specific usability. That's what I mean with my suggestion: It would set the base here in core and the clients would utilize on that depending what's needed for their platform in case of user friendliness.
But I guess that all is currently not planned. On irc NikolaiToryzin told me,
DNS Discovery is the future and the past of mapping names to IDs.
So this is the Standart Specificaion I found about that: https://github.com/Tox/Tox-STS/blob/master/STS.md#dns-discovery and this seems to be already implemented in toxcore's new_api branch. (https://github.com/irungentoo/toxcore/tree/new_api/toxdns) But this was already discussed in this thread before (15th January) and it doesn't reach the level of user friendlyness you both ( @Mosrite , @srkunze ) described. It looks like it doesn't even support phone numbers at all...
Both have the following problems: they are mostly just focused on their platform
While this is certainly true for XMPP this IMHO doesn't apply to SIP. You can use SIP on desktop and mobile without problems. I think newer Android versions (4.1+ ?) even have native SIP support.
Tox could combine everything:
multiplatform secure user friendly
That's what I understood as the goal of Tox
That was my understanding, too. But this means that Tox needs the usability level of Whatsapp, otherwise there is no point for probably 95% of users to try a different service/app. Unfortunately, privacy and security aren't reason enough to switch messaging habits for the majority.
If you want to be found by your name or your number, I don't think you'd have a problem with that fact that the server has your number and or name as plane text, per sé. But if you want to be found by either name or number, and hence provide both, you might want to prevent others from looking up your phone number by only having your name. This is issue 1. Issue number 2 is fakes. With whatsapp you know that if you add my number, you add me. If anybody can register with any number without verification, that would be suboptimal. I know, there is another ticket about authentication, but still, ideally it works without it. As userfriendly as possible. As easy as possible, as quick as possible. Without casing costs for verification. Issue number three is linkability. Onions were introduced such that DHT peers cannot find out who talks to whom. A central server seeing lookupss and ip's sounds suboptimal. Which is almost the same as issue 4: Ideally it would all be decentralized. No single ppoint of failure.
Finding a lookup that takes into account these 4 points (at leats) is what this issue thread is about, IMHO. And this is in no way trivial.
Re 1) With multi-id support, you could easily prevent that, by linking one id to name, the other to number.
Can the ideas here be revisited?
@ProMcTagonist based on @stendler summary (great thing, thanks a lot), ideas are not really useful, since they don't really bring what is needed, and could potentially lead to making network vulnerable to DDoS. As for @stendler idea, I didn't read it all, but even then it seemed to be unworkable due to how Tox network works, i.e. DHT doesn't store Tox IDs in it, nor even your permanent public key.. (and storing plain IDs is not going to happen)
With that being said, please correct me if I'm wrong.
It's been awhile and thinking back, I'm liking the idea of web of trust more. The goal is making the discovery of users you know easier, why not have the list created by the people you have already added?
For example: A adds B using manual tox ID B has a contact list of 30 friends who have each opted in for sharing their ID and "extra data" with friends of friends. A searches for D (nickname Delta) and shows up with 1 friend knows Delta.
To implement
Benefits:
It has a underlying trust (3 people know n) I'd much rather 20 other people know I know F then have my id on some unknown server or centralized list with my phone number.
There's the initial problem of the "1st friend" since that will require a manual Tox ID. Isolated cases will also need manual Tox ID's. Also, should a friend both have a nospam changed and share contact with friends friends checked, it should automatically update the toxid.
I like your proposal. It reminds me of Facebook friends.
Isn't this how freenet works?
I think Retroshare use very similar system to your offer @Mecvak , without a problem.
@Mecvak's retroshare feature could work well if there was a discovery area showing friends of friends, maybe in the Add Friend area
I have quite some friends from the old days with whom I share no friends I still have contact too. So if I want to find those, a "friends of friends" approach does not help. However, a general lookup à la skype or facebook would work in that case.
phone books have a long tradition, and if it is optional, I see no big privacy concerns with it. Ideally you'd want to make it easy to lookup tox id by name or phone number, and a bit harder to do it the other way round (reverse lookup).
The actual problem with this in a decentralized setup is, fake/malicious entries.
I have quite some friends from the old days with whom I share no friends I still have contact too. So if I want to find those, a "friends of friends" approach does not help. However, a general lookup à la skype or facebook would work in that case.
Which as mentioned, would require a manual adding. But it would help with any friends you know who know each other. I don't think I'm way off when guessing that most people have friends who know each other, whether it's work, school, groups, etc. This isn't so much as retroshare's feature (which is a nice validator!) as the original intention is the good ol' fashioned web of trust. General lookup suffers from many problems that's been discussed to death here and on other threads. The web of trust style is decentralized, protects against fake entries and can be easy to use. The inability to easily add someone who's not a friend-of-friend doesn't invalidate that.
Which as mentioned, would require a manual adding.
All addings should be manual. I am against automatic adding of contacts
The inability to easily add someone who's not a friend-of-friend doesn't invalidate that.
No, it does not.
I like the web-of-trust principle, though I wouldn't allow my 'friends', which actually are just contacts, to find who I talked to on Tox. I can see a lot of privacy issues with that. In any case one should be able to decide who sees what contacts. This is almost like sending a Tox messages with the Tox IDs of the contacts you want to share, which can be done today. Perhaps the message can be standardized in some way so that clients can right-click a contact and select "send contact id to ..." and the other end can pick it up with a proper dialog, logging its origin.
By the way, at the ring.cx project they are working on a DHT directory similar to the one Skype has: https://blog.savoirfairelinux.com/en/2015/research-uqam-opendht-ring-project/ They claim it can still be anonymous. But you are right about fake entries, I guess ZRTP or the millionaire protocol could eliminate these.
All addings should be manual. I am against automatic adding of contacts
Should of worded that better. By manual I meant taking a tox id (or shortened one) to add someone as opposed to clicking "add alice?" in some friend discovery place
I like the web-of-trust principle, though I wouldn't allow my 'friends', which actually are just contacts, to find who I talked to on Tox. I can see a lot of privacy issues with that. In any case one should be able to decide who sees what contacts.
Absolutely. I'm not arguing that there is both perfect privacy and ease of discovery. My angle is the scope of the privacy - limited to friends of friends and not the whole world®. On the web, that's a important distinction. Your privacy "surface area" is reduced - not eliminated.
Ring looks intriguing. I'll have to read how it works
Ideally you'd have all three: publically discoverable, friends of friends only, and no discovery at all. And the user can decide between the three. But that might be too complex...
A good way to give users those options would be simply to add someone like a "public, protected or private" mode for their settings. There's no need to really explain that, except perhaps protected which could simply be listed as "Visible to friends of friends"
On Tuesday, November 17, 2015, aaannndddyyy notifications@github.com wrote:
Ideally you'd have all three: publically discoverable, friends of friends only, and no discovery at all. And the user can decide between the three. But that might be too complex...
— Reply to this email directly or view it on GitHub https://github.com/irungentoo/toxcore/issues/1222#issuecomment-157410483 .
Best and server independent way still would be dht option to announce myself(default off) if i wanted and then it can be searced by others like Skype was before Microsoft. Against man in middle would be some sort of confirmation id like ZRTP.
I could not find anything useful on this topic, thus this thread.
Recently, I introduced Tox to my girlfriend and sent her my ToxID. She try it out on her tablet. She was even more surprised regarding the long identifier as she discovered that she even need to scroll to the right to see it fully.
Long story short, she liked the encryption, group, voice and video features, but those IDs... she compared them to "useless ICQ numbers". I know technically that is not quite the same, but from what I know Tox aims to be as user-friendly as it can be.
User discovery is an important part here as well. Is there a way to query after user names? I know they are not unique but it would be a start and if there are two users with the same username, e.g. the last two digits of the ToxID can be used (and interchanged manually) in order to distinguish the buddy from other guys.