search

marcovelon / tdesktop

Telegram Desktop messaging app without end-to-end encryption
https://desktop.telegram.org/
Other
11 stars 2 forks source link

[Feature Request] Secret Chats in Telegram Desktop #2

Open marcovelon opened 3 years ago

marcovelon commented 3 years ago

This issue is a continuation of https://github.com/telegramdesktop/tdesktop/issues/871 and https://github.com/telegramdesktop/tdesktop/issues/16835 and is dedicated to the code bounty campaign related to lack of implementation of a Secret Chat feature (end-to-end encryption in private messages) in Telegram Desktop.

The objective is to implement the Secret Chats feature into this Telegram client: https://github.com/telegramdesktop/tdesktop

It is possible to do it with 3 different ways (but not limited to):

This code bounty doesn't limit the programmer to any specific way of implementation, as soon as the final result will make possible to use Secret Chats on the open source Telegram Desktop (tdesktop) client on Linux and Windows in the exactly same way as it's done in mobile clients.

UPDATE 04/2022 / Funds distribution scheme:

There will be 3 payouts based on completion of the following stages:

The developer(s) will receive 1 ETH for completing one of the stages from the list above, totalling 3 ETH for all stages. The source code should be available and compilable.

The adjacent functionality such as deleting and configuring Secret Chat options can be done during any phase of the work progress described in the list above, however it must be done before or with 3/3.

I have created a verified signature for my Ethereum address containing the funds for this issue: https://etherscan.io/verifySig/4431 (https://etherscan.io/address/0xd19ee4a49b9214c4c22694bb01f225baf35f6efc)

Any voluntary donations are welcome. You can send them to the address above.

My email for communication is marcovelon@protonmail.com

CURRENT PROJECT FUNDING:

The overall funding now is 13000 USDC and 1.18 ETH (~3929 USD)

marcovelon commented 3 years ago

The campaign has been funded with 2.2 ETH which is equivalent to 0.15 BTC (turned out it's impossible to fund campaigns with BTC on Gitcoin). The proof of funds is available in the invoice attached to the campaign available at https://gitcoin.co/issue/marcovelon/tdesktop/2/100026398

UPDATE 04/2022: the campaign has been withdrawn from Gitcoin because their platform is totally broken. The funds distribution will be performed manually by me after completion of steps written in the first post. The current funding is 3 ETH and doesn't limit this only to 1 coder.

OMTDesign commented 3 years ago

I just want to thank you on your efforts to implement secret/encrypted chats into telegram desktop. This feature has been missing since 2015, and has been asked for regularly by the community since. Yet the developers express no interest and close any issue, referring back to a post from 2015.

I wish you and the community well and hope this functionality is implemented.

basedcookie commented 3 years ago

Update: currently working on adding in the logic for the secret chat, GUI is almost finished. Sometime in the next week or 2 I will be uploading my local commits to my account so the code will be available. Mods who see this, let me know if the inactivity issue has been solved, I forget we have to post regular WIP on here.

arch-btw commented 3 years ago

Thank you so much for working on this @basedcookie.

When it's done maybe you can post a crypto address to donate to (BTC, XMR or LTC for example), because I can't for the life of me figure out how gitcoin works.

marcovelon commented 2 years ago

Since the bounty has been abandoned by basedcookie (no updates for more than 2 months), it is now open for a new applicant.

If you are interested to code this feature, you can enroll here: https://gitcoin.co/issue/marcovelon/tdesktop/2/100026398

amritkumarj commented 2 years ago

I had started working on this project, and will soon be posting the updates.

OMTDesign commented 2 years ago

Sad to see them @basedcookie leave. But good to see there is still interest in this project and hopefully this feature will be implemented.

amritkumarj commented 2 years ago

@OMTDesign Yeah, as now I am working on this project it will very soon be completed, I have expertise in cryptographic algorithms and hence it will be not a very difficult task for me

arch-btw commented 2 years ago

@DhruvJain1122 which approach are you taking? Migration to TDLib?

basedcookie commented 2 years ago

@arch-btw can't speak for them, but back when I was working on this I was trying to implement the feature directly into the existing code, personally that proved to be a little challenging and time consuming. If I were to work on it again I would take the approach of migrating to TDLib

marcovelon commented 2 years ago

Alright, I think many of you will agree with the fact we have reached some point of absurd never seen before in FOSS community and it feels like Telegram purposely instructs their open source relations team to avoid this topic with an utmost effort.

I am not sure what exactly is causing a nearly zero interest in this issue, however my suspections are that it's either one of those:

Desktops are less personal. We log in from many places on web and desktop. Many of our desktops are monitored by network administrators at work, or accessed by family members at home. We often leave our desktops and even laptops unattended. It's much easier to keep an eye on your phone. [Telegram team's argumentation]

No, they are not less personal; laptops and towers can be even more personal than mobile devices. On desktop apps we log in with the same frequency as on mobile due to same session-preserving mechanisms. Most people don't let network administrators or family members to access their personal desktops, but in case they do, they should be aware of security risks and take measures to avoid personal data leak (why not writing about this instead of making misleasing statements?). We often leave our mobile unattended and many people don't even use proper security locking mechanisms on Android phones; desktop users are covered by automatic screen locking and login screen; many desktop users also use full disk encryption. It's also easy to keep an eye on your desktop in case it's a laptop, and tower desktops should be protected by full disk encryption. Additionally, the probability of having a mobile device stolen/lost is significantly higher than for desktop. [My counter-argumentation]

Open systems vs. Sandboxed systems. Desktop systems and browsers are much more open environments, as opposed to sandboxed environments like mobile OSs. To have Secret Chats, you need your app to store data on the device, and implementing this is slightly more complicated in open systems. The Web and Desktop apps currently get all data from the cloud on startup and discard it when you quit or log out. This would mean losing all your secret chats each time you shut down your computer. [Telegram team's argumentation]

The sandboxing on mobile devices is as insecure as desktops without sandboxing ("open systems"...); there is no storage sandboxing on mobile devices and all the apps have access to the same storage, since one app can access the files shared by/with another app located on the emulated storage; also, there are lots of efficient sandboxing for desktop, such as firejail or using VMs/containers, which is much more secure than mobile OS sandboxing solutions. To have Secret Chats, you need your app to store data on the device, and implementing this is no way more complicated on desktops, because this is how most programs (or "apps") work since the hard drive technology was created; on desktop devices, especially running Linux with encrypted home partitions or full disk encryption, there is a thing called 'local storage' which is intended to be used for such cases; this is how browsers and other network apps store all the sensitive information as well, and nothing makes tdesktop more special than a regular browser to think or implement a different solution for such a simple task as this. The Web and Desktop apps currently get most data from the local storage on startup and don't discard it when you quit or log out, so there is no point to mislead users stating the otherwise; cloud clients synchronise the data with the local storage in most use cases. This means you shouldn't lose your secret chats at all because they should be stored on the local storage, in the same way as other desktop applications do. [My counter-argumentation]

A chaos of chats with the same person. If each chat only connects two devices, users get many different chats with the same person. Imagine two people that have a laptop, a desktop at work and a mobile device each. That’s 9 secret chats in total, and most of them are useless at any given moment. [Telegram team's argumentation]

This is not a problem at all because people who use Secret Chats on mobile devices have exactly same issues when changing their phones, so they are aware of that. Also people who use E2EE chats are mostly aware of how they work and should understand that encryption keys are always stored on the local storage. Again, this is how most other desktop apps worked for decades and making a "problem" of it seems to be a misleading and unprofessioal. [My counter-argumentation]

Search-related troubles. There’s an added twist when it comes to finding a certain message. Since no server-side search for messages is possible in the E2EE paradigm, the user would have to remember on their own where to look for a particular message (was it in this chat on this device, or in that chat on that device?). Telegram wouldn’t be able to help in any way. [Telegram team's argumentation]

It's possible to search messages in Secret Chats on mobile. E2EE paradigm use client-side search for messages and this is how search is supposed to work for Secret Chats. Users don't need Telegram's help for that, only the possibility to search messages locally, which is already implemented. [My counter-argumentation]

Temporary sessions. And if that wasn’t enough, desktop sessions tend to be a lot more transient. One of the important features of secret chats is that they are destroyed on your device when you log out. Most desktop users log out rather frequently. Telegram Web users do this even more often. [Telegram team's argumentation]

Desktop sessions work in a same way as mobile sessions. Secret chats are destroyed on your device when you log out and this is why they are secret, nothing wrong about it. Most desktop users don't log out frequently and there is a big lack of context in the original statement. Telegram Web users should know better why they use Web client, it's their own choice, however, all this is not a valid argument to not implement secret chats on Web client either, since web-browsers are even used to store cryptocurrency (Metamask and such), and there is no problem with that for people who are aware of security risks; there is no problem enlightning people on security risks associated with web-browsers usage, this is what Telegram team should do instead of writing lots of misleading reasons to not implement Secure Chats. [My counter-argumentation]

I understand that most regular users recklessly follow their misleading arguments because Durov positions himself as man of knowledge in programming, while he is not, and everything he has done so far was outsourcing programmers, not even mentioning how his brother Nikolai Durov coded everything for Pavel during startups and never continued to help him because of marketing oriented ideology of Pavel (I know this from Nikolay himself from the personal conversation with him some years ago). But what's more disappointing, is that even young developers from Telegram open source team, such as anonymous guy john-preston do share this weird ideology, for some reason supporting everything written on their page explaining "Why are Secret Chats only available on their devices of origin".

Just look at this:

john-preston commented on Jun 23, 2020 @dm17 I say that PR with a secret chat option won't be merged because I can't imagine anyone to produce such PR in such quality that it could be merged. It is a very big task that will have changes in almost all of the application logic and a huge amount of UI changes that should be discussed with the designer of the current interface, so I just don't see any reason to discuss the PR of secret chats in tdesktop — no one will do it.

For example, multiple accounts support is a much easier feature and it has a much less UI integration and still no one PR-ed it (and no one will, but I hope to implement it some day).

See how many times this guy used the wording "no one", supposing that "no one will do it" and even if someone will, he won't merge it. It seems that we are dealing with some amateur frontend coder who doesn't understand cryptography or has no experience with coding practical client/server applications with PKI and who is assuming that just because he doesn't know how it can be done, no one else does. So, based on his logic, there are no programmers in the entire world who can code a simple client-side encryption.

I know that this guy has triggered skepticism from many experienced and knowledgeable people in this topic who instead of trying to prove him wrong, decided to not waste their time and just to leave Telegram. Some people even tried, but Durov often refute such stuff with well-marketed sophistry that results to work on his major part of the audience, since most of them don't have any knowledge in infosec and prefer believing what he says just because they trust him.

Another reason behind all this hesitance on this issue could be that they are afraid of letting too much users to have conversations out of their control, ensuring that they can provide users communications logs to the law enforcement when asked, since their normal "cloud" chats permit to do that easily. Actually, the increased amount of censorship in their platform lately indicates that it could be true.

Anyway, I am stopping to try finding only 1 coder for this task via non-efficient platforms such as Gitcoin (whose most important features stopped working completely and devs don't have any ETA to fix their stuff) because I think it's better if more people worked on this issue as supposed to be in the open source community. I have asked to refund my funds from Gitcoin, which was done, and from now will watch the contributions made by particular devs to this issue. Once this task is complete, I will distribute my cryptofunds to each developer based on their amount of work, which will also allow others to donate to them. We could also try to launch the campaign on https://tallyco.in purposely for funding this issue. My Ethereum address verification with the funds prepared for payout on this issue: https://etherscan.io/verifySig/4431

If you know how we can organize this better, please let's discuss it.

p.s. @arch-btw he became inactive on this issue since the first week of his application

@basedcookie thanks for the feedback, and while I understand that this task may have looked challenging to you and this is normal even if your coding level is good - simply because encryption is rarely employed in C/C++ coding tasks, but I still believe it's a relatively easy task for programmers who have specific experience with coding encryption algorithms on C/C++. I also think that the lack of interest for this issue by people with experience in this field is caused by Telegram's shady policies on E2EE encryption, described by me earlier.

dm17 commented 2 years ago

@marcovelon - good eye! Yes, I just think it is too difficult for them to get a well hidden and functional backdoor into the secret chat functionality on FOSS desktop. In this age of information warfare, I think it is lazy not to assume such things. If a Russian company can re-domicile as Emirati and then censor RT, way ahead of a war that ends up using Telegram as a primary information source on the opposing side... This is going to get complicated to analyze. Of course I love Telegram - the engineering marvel - but intentions factor in when we have to spend years guessing about their insistence to avoid E2EE on FOSS desktop while mobile can afford two full implementations from scratch (Telegram and Telegram X)... It begs the context of mobile being more secure. Yes, phones are more secure - with respect to their own root-of-trust! Desktops - despite these black box operating systems running transparently with respect to the host OS called "management engines" - have still not removed root-of-trust from each user's control as effectively as mobile solutions (where the user is sandboxed away from having full permission and even ownership of their own hardware). The context of mobile being more secure is: mobile is a more secure and foolproof eavesdropping platform.

marcovelon commented 2 years ago

There are still many variables that can make phones less secure, for example users themselves, the OS being used, proprietary hardware with same hidden surprises similar to ME/PSP and so on. For example, one laptop with soldered RAM running coreboot and Debian is already more secure than most mobile devices running stock Android or iOS vulnerable to Pegasus, while one mobile phone running security-hardened AOSP with F-Droid apps only can be more secure than most desktops running Windows 10, but in any case this is already a completely another subject.

The main point is, Telegram's team have invented all this really weird argumentation as an excuse for not implementing Secure Chats on tdesktop, and that makes this whole story comical. And note how they don't even mention mobile device vs desktop platforms security, they just say that 'X is less personal than Y', which is already very vague and pointless, whatever it means.

dm17 commented 2 years ago

@dm17, good remark. I also understand your point behind mobile devices being more secure and while what you said is valid, there are still many variables that can make phones less secure, for example users themselves, the OS being used, proprietary hardware with same hidden surprises similar to ME/PSP and so on. For example, one laptop with soldered RAM running coreboot and Debian is already more secure than most mobile devices running stock Android or iOS vulnerable to Pegasus, while one mobile phone running security-hardened AOSP with F-Droid apps only can be more secure than most desktops running Windows 10, but in any case this is already a completely another subject.

The main point is, Telegram's team have invented all this really weird argumentation as an excuse for not implementing Secure Chats on tdesktop, and that makes this whole story comical. And note how they don't even mention mobile device vs desktop platforms security, they just say that 'X is less personal than Y', which is already very vague and pointless, whatever it means.

They'll never grant you that anything can be as secure as mobile - because as I described it cannot. This is because they have a different definition of security than you or I. Their definition is: how securely can eavesdropping be executed? Smartphone security systems are far more ideal for that. I'm not implying that they all believe this or understand this, but the "thought leaders" that they must follow to have jobs all seem to enforce this view.

marcovelon commented 2 years ago

They are not in a position to "grant" that to anyone while their project still has some open source community support. They would have to go fully proprietary for enforcing such views you described. Currently, they already are facing reputational losses due to more and more incoming gag orders they have to attend, thus neglecting the open source community will make them uncompetitive. Honestly, I am quite happy with truly secure platforms like Matrix, xmpp with OMEMO or Briar. The only reason why I personally have to deal with Telegram is some significant amount of contacts that find it hard or impractical (for them) to use anything more secure, and I have to be glad that they at least don't force me to use Whatsapp or other proprietary chatting clients, so I can make encrypted chat with them using a phone. Well, I admit that reading some channels is nice too, but it's manageable to live without them by substituting them with a personal collection of quality RSS feeds.

Anyway, let's give it a last chance trying finding coders for this issue, otherwise it seems that secure chats on tdesktop are never going to happen.

Tpimp commented 2 years ago

@marcovelon I'm interested And I'm just about done with my current project.

I've looked at the effort to add TDLIB to tdesktop. A huge amount of the tdesktop codebase is making QWidgets look good. As a Qt developer, I've stopped burning time on that long ago. Instead of shoehorning TDLIB into tdesktop, I think a fresh application should be written around the TDLIB library. I looked at Cutegram and in many ways its just way too much Qml to run smoothly and error free. Qml should be used to describe the UX and everything else (business logic and lower) should be raw performant C++.

Both avenues of using mtproto or generating a qml api from the schema (cutegram) seem less likely for long term success than just using TDLIB ( a choice that might not have been available at the start of these other projects).

Anyhow if there is enough interest I'd be willing to start a new project for a modern telegram desktop client. Benefits of new client

I think a solid 4-6 months to reach 1.0 (working full time). Maybe a crowdfunding campaign? Or we could start it and just let people show up naturally and contribute over the next couple months/years. Anyone wanting to chat more about funding or collaborating on a project can find me on Device Chain discord.

dm17 commented 2 years ago

@marcovelon I'm interested And I'm just about done with my current project.

I've looked at the effort to add TDLIB to tdesktop. A huge amount of the tdesktop codebase is making QWidgets look good. As a Qt developer, I've stopped burning time on that long ago. Instead of shoehorning TDLIB into tdesktop, I think a fresh application should be written around the TDLIB library. I looked at Cutegram and in many ways its just way too much Qml to run smoothly and error free. Qml should be used to describe the UX and everything else (business logic and lower) should be raw performant C++.

Both avenues of using mtproto or generating a qml api from the schema (cutegram) seem less likely for long term success than just using TDLIB ( a choice that might not have been available at the start of these other projects).

Anyhow if there is enough interest I'd be willing to start a new project for a modern telegram desktop client. Benefits of new client

* We start with the design first and can base it on modern Telegram application styles and techniques.

* The UX is in Qml so things that took months to build in visuals for tdesktop, instead take days.

* The code base and architecture follow a strict rule of UX is in Qml, backend in C++. Making development by future developers easier.

* Support everything we can get out of TDLIB, including [secret chats](https://core.telegram.org/tdlib/docs/classtd_1_1td__api_1_1secret_chat.html)

* Clear packaging/deploy strategy to provide appImages, flatpaks, and even package manager (.deb/.rpm) packages

I think a solid 4-6 months to reach 1.0 (working full time). Maybe a crowdfunding campaign? Or we could start it and just let people show up naturally and contribute over the next couple months/years. Anyone wanting to chat more about funding or collaborating on a project can find me on Device Chain discord.

If you wrote it from scratch, then why make it telegram-specific anyway rather than a Pidgin-like app that supports Telegram? Anyway, I refuse to believe the easiest way to get secret chats working on tdesktop for linux is rewriting it from scratch.

Tpimp commented 2 years ago

I think scope is the big reason for not supporting every different messaging protocol, design something that does telegram really well first....

Maybe you're right and it is easier. But I just spent nearly 2 hours tracing through the tdesktop codebase and I'm dizzy. The code itself is fine but there is a lot of QWidget specific issues all throughout and I don't feel that's a good use of a developers time as QWidgets are always going to requires tons of work to build modern UX.

Also because from my understanding the project must simultaneously change its underlying logic to use TDLIB (not just for secret chat but to migrate the other Telegram features as well) you end up with a LOT of changes. Sometimes when changing that much code on a project this big and old, it is simply a nightmare to test regressions alone.

A new clean slate prevents many of those problems and replaces them with the requirement to do some things over. Id rather participate in a fresh effort with a reduced codebase size... That is just my two cents, I could be wrong.

dm17 commented 2 years ago

Strange to think you can redo Telegram and do better than its creators, considering it is a world-class UX (yes I realize you're not redesigning the UX but even replicating a UX entirely is difficult, no?). Furthermore, Telegram X redid it from scratch with a much larger support/funding and still hasn't accomplished all of the features of the original - and hasn't gotten as many users. Doesn't that put the rewrite into perspective? Why are all the complicated features easy to bring to tdesktop for Linux except end-to-end encryption, which actually is totally FOSS (because it is two clients connecting and clients are open source). Versus client-server, which the telegram server is closed source.

Tpimp commented 2 years ago

The design and look of the application isn't being debated... but I've rewritten entire QWidget projects in qml in a fraction of the time and with better results. Google around the many projects that have done this over the last 5 years. It's like I would have to explain the logic and advantages of higher level languages.

If the attitude is such that a new effort is discouraged and mentioning of it will make people triggered and cranky, that's fine. Im looking for fun projects to work on not to change people's attitudes about software. Best of luck!

dm17 commented 2 years ago

Not cranky at all; just wondering how you would see my questions... Figured hard questions would be useful gates for starting such a large undertaking... Especially the last one about this merely being a client<->client feature we are missing (and clients are FOSS). Not only are you free to do as you please, but myself and others in this thread would like to see you attempt it.

Tpimp commented 2 years ago

It's a big undertaking. I can't imagine it happening with a single individual alone.

I would be interested in joining an effort or team. But I think many have moved on from telegram. Even the developer of cutegram seems to have moved on for "Oxygen". And I'm personally more interested in the technology of the matrix protocol myself.

The single reason I would consider doing a Telegram client is because as far as I can tell there is not a Qt6 TDLib client yet. I see someone asked about linking issues on the Qt forums late last year but it could have been someone from this thread attempting to link it into tdesktop.

In theory I could help build the first modern Qt6 Telegram desktop client using the "modern" Telegram TD library with all its bells and whistles. But I'd have to really be into Telegram to do that. Would people use it? Bigger question all together. I shouldn't have suggested you're cranky. I get you just want what seems to be a low hanging fruit of a feature in an application you already use and someone suggests "I build new application..." - I'd be annoyed too. 😄

marcovelon commented 2 years ago

@Tpimp thank you for manifesting interest in this.

Your proposal on starting a new Telegram client sounds interesting, but since this issue here is centered only around one specific feature, as also noted by @dm17, it seems like the current funding and organization here won't be sufficient for a scale of your proposal. Personally, I am not ready to fund a whole new project at this time, but would surely consider sponsoring some part of it if there was a crowdfunding campaign that accepted crypto (I suggest tallyco.in). I am convinced that with your great public portfolio and experience you may be able to bring sufficient funding on that, if you created a well-formed campaign exposing a full project plan.

Meanwhile, I am considering delisting Migrating tdesktop to TDLib from possible solutions present in this issue, because:

  1. funding for migrating the whole client to TDLib here is probably insufficient due to time requirements for amount of that work
  2. this brings nothing else but more confusion, because devs start looking for it in a first place assuming it's the easiest solution, ending with full understanding of how many rewrites will it require after reviewing the current telegram-desktop code.
  3. since we are working in a scope of the fork aiming for making a PR with Secret Chats to telegramdesktop/tdesktop, there is no point to migrate it to TDLib while telegramdesktop/tdesktop is not using it.

To make it pragmatic, we just need to implement a few client-side cryptographic methods fully specified by the official documentation. This requires minimum UI coding and most of its work is implementing the core functionality.

What can be done with TDLib in the current situation is trying to include it in telegramdesktop/tdesktop and adding Secret Chats with calls to its API, essentially for DH key exchange and encrypting/decrypting text and media.

lukejohnsonrp commented 2 years ago

I have dedicated a few hours to dig into particular libraries that provide support for E2EE chats in Telegram to determine complexity of this implementation and concluded that its difficulty referred by the official devs is being exaggerated.

This report will provide a better perspective of what needs to be done here.

vysheng/tgl

Reviewing https://github.com/majn/telegram-purple code in order to discover how secret chats are implemented with tgl, because it still has secret chats working.

Creating secret chats

Requesting secret chats

tgl_do_create_encr_chat_request [tgl/queries-encrypted.c] => 
get_dh_config_on_answer [tgl/queries-encrypted.c] =>
bl_do_set_dh_params [tgl/binlog.c] =>
tgl_do_send_create_encr_chat [tgl/queries-encrypted.c] => 
[state = sc_waiting] =>
bl_do_encr_chat [tgl/binlog.c] =>
send_encr_request_on_answer [tgl/queries-encrypted.c] =>
tglf_fetch_alloc_encrypted_chat [tgl/structures.c] =>
[wait for peer to accept] =>
tglu_work_update [tgl/updates.c]  =>
tglf_fetch_alloc_encrypted_chat [tgl/structures.c]=> 
[state = sc_ok] =>
bl_do_encr_chat [tgl/binlog.c] =>
tgl_do_create_keys_end [tgl/queries-encrypted.c]

Accepting secret chats

tglu_work_update [tgl/updates.c] => 
tglf_fetch_alloc_encrypted_chat [tgl/structures.c] => 
[state = sc_request] =>
bl_do_encr_chat [tgl/binlog.c] =>
tgl_do_accept_encr_chat_request [tgl/queries-encrypted.c] => 
get_dh_config_on_answer [tgl/queries-encrypted.c] => 
bl_do_set_dh_params [tgl/binlog.c] =>
[we accepted it and send response] =>
tgl_do_send_accept_encr_chat [tgl/queries-encrypted.c] => 
[state = sc_ok] =>
bl_do_encr_chat [tgl/binlog.c] =>
send_encr_accept_on_answer [tgl/queries-encrypted.c] => 
tgl_do_send_encr_chat_layer [tgl/queries-encrypted.c]

Sending secret chat messages

tgl_do_send_message [tgl/queries.c] =>
bl_do_edit_message_encr [tgl/binlog.c] =>
tgl_do_send_msg [tgl/queries.c] => 
tgl_do_send_encr_msg [tgl/queries-encrypted.c]
encr_finish [tgl/queries-encrypted.c]
encrypt_decrypted_message [tgl/queries-encrypted.c] => 
TGLC_aes_ige_encrypt [tgl/crypto/aes_altern.c] => 
do_ige_encrypt [tgl/crypto/aes_altern.c] => 
gcry_cipher_encrypt [libgcrypt]

Receiving secret chat messages

tglu_work_update [tgl/updates.c] => 
tglf_fetch_encrypted_message [tgl/structures.c] => 
decrypt_encrypted_message [tgl/structures.c] => 
TGLC_aes_ige_encrypt [tgl/crypto/aes_altern.c] => 
do_ige_decrypt [tgl/crypto/aes_altern.c] => 
gcry_cipher_decrypt [libgcrypt]

The whole implementation is quite minimalistic and straightforward. Reading its code gives full understanding of how secret chats work.

For both encryption and decryption the TGLC_aes_ige_encrypt wrapper function is used, that either call do_ige_encrypt or do_ige_decrypt for invoking gcry_cipher_encrypt / gcry_cipher_decrypt functions from libgcrypt

Currently it may crash on incoming encrypted messages, but it's not because it can't decrypt them. It looks like something related to mtproto protocol upgrades, but reviewing it wasn't within a scope of my objectives. https://github.com/majn/telegram-purple/issues/567

tdlib / C++

Reviewing tdlib-purple code in order to discover how secret chats are implemented with tdlib.

tdlib-purple makes full use of abstraction that tdlib offers. No events related to encryption and decryption happen in the plugin itself, and for that reason we should observe tdlib.

Creating secret chats

Requesting secret chats

td_api::createNewSecretChat =>
MessagesManager::create_new_secret_chat =>
SecretChatsManager::create_chat =>
SecretChatActor::create_chat => 
SecretChatActor::do_create_chat_impl =>
[state = State::SendRequest] =>
SecretChatActor::run_auth => 
DhHandshake::get_g_b / auth_state_.handshake.get_g_b =>
send_net_query / QueryType::EncryptedChat / telegram_api::messages_requestEncryption / auth_state_.handshake.get_g_b =>
[auth_state_.state = State::WaitRequestResponse] =>
SecretChatActor::on_update_chat / telegram_api::encryptedChat =>
DhHandshake::set_g_a / auth_state_.handshake.set_g_a =>
DhHandshake::gen_key / auth_state_.handshake.gen_key => 
SecretChatActor::calc_key_hash =>
[auth_state_.state = State::Ready]  => 
[state = SecretChatState::Active]

Accepting secret chats

UpdatesManager::on_pending_updates =>
UpdatesManager::on_update / telegram_api::updateEncryption =>
SecretChatsManager::on_update_chat =>
SecretChatsManager::flush_pending_chat_updates =>
SecretChatsManager::do_update_chat =>
SecretChatActor::check_status =>
SecretChatActor::on_update_chat / telegram_api::encryptedChatRequested =>
[auth_state_.state = State::SendAccept] => 
DhHandshake::set_g_a / auth_state_.handshake.set_g_a => 
SecretChatActor::run_auth =>
DhHandshake::gen_key / auth_state_.handshake.gen_key => 
SecretChatActor::calc_key_hash =>
send_net_query / QueryType::EncryptedChat / telegram_api::messages_acceptEncryption / auth_state_.handshake.get_g_b  =>
SecretChatActor::on_update_chat / telegram_api::encryptedChat =>
[state = SecretChatState::Active]

State updates during key exchange

SecretChatActor::run_auth

SecretChatActor::send_update_secret_chat

SecretChatActor::on_outbound_action / secret_api::decryptedMessageActionNotifyLayer

SecretChatActor::on_inbound_action / secret_api::decryptedMessageActionNotifyLayer

Sending secret chat messages

MessagesManager::do_send_message =>
MessagesManager::on_text_message_ready_to_send =>
MessagesManager::send_secret_message =>
SecretChatsManager::send_message =>
SecretChatActor::send_message =>
SecretChatActor::send_message_impl =>
SecretChatActor::create_encrypted_message =>
Transport::write => 
Transport::write_e2e_crypto =>
Transport::write_crypto_impl => 
Transport::calc_message_ack_and_key / Transport::calc_message_key2
KDF / KDF2 [td/mtproto/KDF.cpp]
aes_ige_encrypt [tdutils/td/utils/crypto.cpp]

Notice how Transport::write_e2e_crypto for secret chat messages and Transport::write_crypto for cloud messages share exactly same cryptography, implemented in Transport::write_crypto_impl. For more details look up Transport::write

Receiving secret chat messages

UpdatesManager::process_qts_update / telegram_api::updateNewEncryptedMessage or UpdatesManager::process_get_difference_updates / telegram_api::EncryptedMessage =>
SecretChatsManager::on_new_message =>
SecretChatsManager::add_inbound_message => 
SecretChatActor::add_inbound_message =>
SecretChatActor::do_inbound_message_encrypted =>
SecretChatActor::decrypt  =>
Transport::read =>
Transport::read_e2e_crypto =>
Transport::read_crypto_impl =>
KDF / KDF2 [td/mtproto/KDF.cpp]
aes_ige_decrypt [tdutils/td/utils/crypto.cpp]
Transport::calc_message_ack_and_key / Transport::calc_message_key2

Transport::read reads the chat for both cloud and secret chats. For cloud chats it calls Transport::read_crypto and sets info->type to PacketInfo::Common For secret chats it calls Transport::read_e2e_crypto and sets info->type to PacketInfo::EndToEnd Both Transport::read_crypto and Transport::read_e2e_crypto call Transport::read_crypto_impl to decrypt messages via aes_ige_decrypt. This means cloud and secret chats share exactly same cryptographic functions, thus they should be already present in Telegram Desktop.

Telegram Desktop / C++

A quick review of the cryptographic functions already existing in Telegram Desktop demonstrates the presence of all the functions required for secret chats implementation.

Since cloud chats use exactly same cryptographic methods used for secret chats, the necessary cryptographic implementation is already present in Telegram Desktop and can be found in the following source code files:

For example, these AES IGE functions wrap the original deprecated openssl AES_ige_encrypt function (openssl/crypto/aes/aes_ige.c):

They are absolute equivalents for wrappers aes_ige_decrypt and aes_ige_encrypt from TDLib.

All the functionality related to DH key exchange is also present in Telegram Desktop, because mtproto use it for transport layer security and cloud chats.

Hence, what should be done to implement secret chats in Telegram Desktop is adding protocol-specific headers, routines and event binds related to secret chats, using cryptographic functions already present in Telegram Desktop.

Speaking of UI implementation:

  1. Add a green lock icon
  2. Add context menu items related to secret chats

And of course there is absolutely no need to migrate Telegram Desktop to TDLib in order to implement secret chats.

Unfortunately not myself a C++ coder, otherwise it would have been done, because it's really just a week of work for an average C++ coder.

marcovelon commented 2 years ago

Thank you for confirming my and others point of view regarding this. I believe your research will help to facilitate the implementation.

gitcoinbot commented 2 years ago

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

Work has been started.

These users each claimed they can complete the work by 2 months, 2 weeks ago. Please review their action plans below:

1) sergiotarxz has been approved to start work.

Hi, I have read from upstream that this feature would not get merged into upstream since the opinion of telegram desktop maintainer is against it, I could implement it but as patches over tdesktop or as a fork which means that once finished my work a mainteinance should be paid to keep the project synced with upstream or otherwise you may choose to maintain the fork yourself, I think the tdesktop posture is now obsolete since there are mobile phones which use telegram desktop so the original complain no longer applies.

Learn more on the Gitcoin Issue Details page.

sergiotarxz commented 2 years ago

Hi, since I am a programmer and not a designer I could benefit from a draft about how the interface should ideally look afterwards.

I looked at the relevant tdesktop ui code and it is hard to work with, but I will hopefully find my way.

marcovelon commented 2 years ago

@sergiotarxz thank you for taking part in this project.

https://github.com/UnigramDev/Unigram could serve as a reference for desktop UI in this specific case, since it replicate the mobile UI logic for secret chats. Here are screenshots that demonstrate it explicitly

"Start Secret Chat" button from the user info tab: https://oshi.at/NmTt.png How secret chat looks in the chat list: https://oshi.at/nczG.png

Basically it follows the logic of how it works in the mobile app. The icons from the mobile version are here:

https://github.com/Telegram-FOSS-Team/Telegram-FOSS/blob/master/TMessagesProj/src/main/res/drawable-mdpi/menu_secret.png https://github.com/Telegram-FOSS-Team/Telegram-FOSS/blob/master/TMessagesProj/src/main/res/drawable-mdpi/list_secret.png https://github.com/Telegram-FOSS-Team/Telegram-FOSS/blob/master/TMessagesProj/src/main/res/drawable-mdpi/msg_start_secret.png

https://github.com/Telegram-FOSS-Team/Telegram-FOSS/blob/master/TMessagesProj/src/main/res/drawable-hdpi/menu_secret.png https://github.com/Telegram-FOSS-Team/Telegram-FOSS/blob/master/TMessagesProj/src/main/res/drawable-hdpi/list_secret.png https://github.com/Telegram-FOSS-Team/Telegram-FOSS/blob/master/TMessagesProj/src/main/res/drawable-hdpi/msg_start_secret.png

To recap:

  1. To start a secret chat: "Start Secret Chat" context menu item in the user info tab and optionally when you right-click on some already created unencrypted chat in the chat list
  2. To view a secret chat: a lock icon near the chat title in the chat list view and inside a chat view near the chat title
  3. To end a secret chat: "Stop Secret Chat" context menu item when you right-click on the already created secret chat in the chat list

Note that you don't need to create a separate UI for secret chats, since secret chats use default cloud chats view and its actions depend on whether the chat type is secret or not (i.e. isSecretChat in Telegram for mobile). Here is a more detailed example of how TDLib flags secret chats from https://raw.githubusercontent.com/tdlib/td/master/td/telegram/MessagesManager.cpp with my comments:

// on_text_message_ready_to_send() is invoked by do_send_message(), they are used for both cloud and secret chats to send messages based on the chat type (DialogType::SecretChat || DialogType::Chat)

void MessagesManager::on_text_message_ready_to_send(DialogId dialog_id, MessageId message_id) {
  LOG(INFO) << "Ready to send " << message_id << " to " << dialog_id;

  auto m = get_message({dialog_id, message_id});
  if (m == nullptr) {
    return;
  }
  if (G()->close_flag()) {
    return;
  }

  CHECK(message_id.is_yet_unsent());

  auto content = m->content.get();
  CHECK(content != nullptr);

  if (dialog_id.get_type() == DialogType::SecretChat) { // right here it checks if it's a secret chat matching against the DialogType::SecretChat flag
    CHECK(!message_id.is_scheduled());
    send_secret_message(dialog_id, m, get_secret_input_media(content, td_, nullptr, BufferSlice()));
  } else { // when previous condition don't match, here it proceeds with sending a cloud (unencrypted) chat which is defined with the DialogType::Chat flag 
    const FormattedText *message_text = get_message_content_text(content);
    CHECK(message_text != nullptr);

    int64 random_id = begin_send_message(dialog_id, m);
    td_->create_handler<SendMessageQuery>()->send(
        get_message_flags(m), dialog_id, get_send_message_as_input_peer(m), m->reply_to_message_id,
        get_message_schedule_date(m), get_input_reply_markup(m->reply_markup),
        get_input_message_entities(td_->contacts_manager_.get(), message_text->entities, "do_send_message"),
        message_text->text, m->is_copy, random_id, &m->send_query_ref);
  }
}
dm17 commented 2 years ago

In terms of UX, why not just checkout the Android app's secret chat functionality, and copy that?

marcovelon commented 2 years ago

Unigram does replicate Android app's UI for secret chats.

sergiotarxz commented 2 years ago

I really thanks all the meaningful data you provided me, I am still getting familiar with the codebase and the c++isms in the code, I hope to start coding really soon.

dm17 commented 2 years ago

Awesome! While implementing this, would you mind keeping an eye out for any potential ways the server side could get the client to give it the decrypted secret chats? Pretty vital for the utility of this feature :)

sergiotarxz commented 2 years ago

@dm17 Do you mean to avoid the server to get the decrypted chats?

dm17 commented 2 years ago

@sergiotarxz Not "avoid" - it absolutely should not be able to access them if it went rogue and tried.

sergiotarxz commented 2 years ago

@dm17 That is a harder than you think petition Telegram Desktop is composed for more of 1400 files with tons of lines written in a memory unsafe language like c++ to audit the entire code to ensure it's safeness is outside of my capabilities.

If you are talking about searching a backdoor for getting the chats from the client it would be also like seeking for a concrete needle in New York.

dm17 commented 2 years ago

@dm17 That is a harder than you think petition Telegram Desktop is composed for more of 1400 files with tons of lines written in a memory unsafe language like c++ to audit the entire code to ensure it's safeness is outside of my capabilities.

If you are talking about searching a backdoor for getting the chats from the client it would be also like seeking for a concrete needle in New York.

Definitely not asking for that (an extensive audit). For example, if the two clients that chat secretly establish their own key, then that key is on each client (and not the server). But how does the client expose secret-chats that are decrypted for reading vs regular chats? There would be designs of secret chat feature that would make the encrypted chats a lot more secure while in the decrypted state, it seems...

sergiotarxz commented 2 years ago

I am doing my first native build of Telegram desktop master taking as example the Gentoo ebuild, I changed "SEND MESSAGE" for "hola" just to check the file I am looking is the file where I have to put the secret chat button.

sergiotarxz commented 2 years ago

Definitely not asking for that (an extensive audit). For example, if the two clients that chat secretly establish their own key, then that key is on each client (and not the server). But how does the client expose secret-chats that are decrypted for reading vs regular chats? There would be designs of secret chat feature that would make the encrypted chats a lot more secure while in the decrypted state, it seems...

I thought if possible to avoid storing these chats in the client unencrypted and fetch them every time it is wanted to read them from the server encrypted, that way allows to have them unencrypted in memory only when they are being read.

sergiotarxz commented 2 years ago

Also, I am using the master branch of upstream for my development, I don't know how updated is this repository but I thought that updating the code to match upstream later would be more difficult that simply starting from upstream.

marcovelon commented 2 years ago

@sergiotarxz: I thought if possible to avoid storing these chats in the client unencrypted and fetch them every time it is wanted to read them from the server encrypted, that way allows to have them unencrypted in memory only when they are being read.

Ideally we shouldn't reinvent security at this point and better stick to estabilished practices. All TDLib desktop apps and Telegram-FOSS (mobile app) use local storage for storing secret chats keys, so should we, because if you try to store them somewhere else (i.e. server) it can pose additional risks instead of improving security, because first of all you will have to encrypt those keys to keep them on a server and this will require a local storage anyway to keep the master key to decrypt them later, thus it's debatable whether there is any point for this at all. Any decent desktop app that use E2EE stores its keys in the local storage encrypted by a master key, because desktops are secure by default when their users follow standard security recommendations. Once desktop security is compromised it won't really matter how secure is approach the app takes to keep the keys.

Now, regarding @dm-17's last point, I don't think it's a good timing to bring this into concern right now, because it will only distract you from the main task. Security audit will be performed by me and @lukejohnsonrp since he offered to help on this per our last brief mail exchange, however, even if tdesktop team decides to implement some rogue code later to compromise our secret chat implementation, this won't be yours or our liability and it seems a bit pointless to try to prevent this stuff, since we will rely on the third party maintainer (john-preston) to keep the app up-to-date anyway. The best we can do is to perform periodic security audits of their updated branch on each merge.

For now I suggest to stick to the plan relevant to this issue:

[1/3] Estabilishing Secret Chats via DH key exchange (ability to send and accept Secret Chat requests) [2/3] Ability to send and receive encrypted messages in estabilished Secret Chats [3/3] Re-keying (PFS) and media sharing in Secret Chats

Using local storage for keys is the ideal solution at this time, because implementing anything else will be time consuming and require additional and unnecessary planning before anything important is done. If we want to implement additional security to this, I suggest to postpone the discussion for later, once the issue-relevant objective is complete.

@sergiotarxz: Also, I am using the master branch of upstream for my development, I don't know how updated is this repository but I thought that updating the code to match upstream later would be more difficult that simply starting from upstream.

This is the best choice because currently their master branch is up-to-date and compiles without issues.

sergiotarxz commented 2 years ago

I achieved to do this, right now it only does the same as the start chat button, but at least I achieved to do my first tdesktop change.

I agree with storing the keys in local storage, although I still don't know how storing data is done I will have to investigate further.

PD: You may have noticed that I used an emoji instead of a icon, that is because I don't know yet how to use a icon, I will come back later to that issue when I am more familiar with the codebase.

sergiotarxz commented 2 years ago

I am a little stuck right now with symbols of this type MTPmessages_GetStickerSet I do not know where they come from I have been ripgrepping the codebase searching for where the magic is declared but I didn't find nothing.

For what I think those kind of symbols are wrappers around the server comunication so I will need to make use of similar symbols like MTPmessages_RequestEncryption, but I still do not know if when I try to use them they will be available or I will have to do something special to declare them.

sergiotarxz commented 2 years ago

I achieved to be able to work with those symbols, they are autogenerated and stored here: out/Telegram/gen/scheme.h, I keep discovering the building pieces. I took a look in how the history is populated, it is a little obfuscated, but I hope to be able to overcome that.

sergiotarxz commented 2 years ago

I found myself needing another phone number to continue testing, so I bought a spain prepaid sim card for 10€, I am stalled meanwhile it arrives. (A small price to pay for 3eth :) )

sergiotarxz commented 2 years ago

I noticed telegram allows you to get entropy from the server, if phones as telegram sugest you to do use it, it would be potentially a cipher undermine as @dm17 asked to look for. I am not a expert in cryptography, but it sounds really suspicious. I am not going to use that server entropy, but if you think this can undermine your conversations with telegram foss mobile users I would suggest to look for that.

Cryptography is as weak as the weakest key involved. :)

sergiotarxz commented 2 years ago

Other issue I wanted to comment is that telegram offers no way to have multiple keys in a E2EE conversation and telegram foss (Android) will auto accept any incoming telegram secret chat as soon as it receives it so if you think phones undermine you security scheme in telegram e2ee you will need to find a way to avoid telegram foss to accept secret chats or start yourself always the conversations with third party, or when you know you are going to receive an incoming secret chat put your phone in airplane mode.

I tested a little with a Android phone and E2EE chats with Telegram Foss, but I don't have a Android phone suitable for testing since I am a PinePhone user.

sergiotarxz commented 2 years ago

Note for me: Change here to process the updates related to encryption https://github.com/telegramdesktop/tdesktop/blob/36b8d03b1d455f9cca8d32ae5b33c419a104d6fa/Telegram/SourceFiles/api/api_updates.cpp#L1475

marcovelon commented 2 years ago

@sergiotarxz if you need a new phone number to register another Telegram account I can handle a virtual SIM service account to you so there is no need to use a physical SIM. Send me an email to marcovelon@protonmail.com so I can send it.

Regarding Telegram FOSS for tests, consider running Android-x86 on a VM in case you have some spare RAM.

sergiotarxz commented 2 years ago

I already mailed you. :)

sergiotarxz commented 2 years ago

I just discovered tdesktop uses only the server provided entropy without any computer side entropy for calls so they are in fact not E2EE and can be intercepted by the server if they wanted so.

I am taking inspiration from calls code.