search

the-djmaze / snappymail

Simple, modern & fast web-based email client
https://snappymail.eu
GNU Affero General Public License v3.0
981 stars 114 forks source link

Improve OpenPGP #89

Open the-djmaze opened 3 years ago

the-djmaze commented 3 years ago

Describe the bug The RainLoop OpenPGP implementation is incorrect.

  1. Encrypt message should be with 'Recipient' public key (not your own key)
  2. Sign a message should be with 'From' private key
  3. Received signed "Content-Type: multipart/alternative" message can't be verified yet (only plain)
  4. etc. etc.

Some limitations are caused by the current implementation in JavaScript. Either the whole message body should be rendered as-is in JavaScript or handled in PHP with https://php.net/gnupg or others.

Reported issues at RainLoop: https://github.com/RainLoop/rainloop-webmail/issues?q=is%3Aissue+is%3Aopen+pgp

I made a Wiki page that explains the rules. https://github.com/the-djmaze/snappymail/wiki/OpenPGP

TODO:

Keys

Import public/private keys

Note: as of v2.34 you can search public key servers to find them, and import all keys from server into OpenPGP.js

View public/private keys

Delete public/private keys

Allow private keys without password

Decrypt / Sign when multiple keys exist

Received messages

PGP/Inline (cleartext)

Decrypt

Verify signature

Decrypt then verify signature

PGP/MIME (multipart)

Decrypt

Verify signature

Decrypt then verify signature

Sending messages

PGP/Inline (cleartext)

❌ no, everything is PGP/MIME

PGP/MIME (multipart)

Encrypt message text

Encrypt attachments

Sign

Sign then Encrypt

Autocrypt

As requested in issue #342 for https://autocrypt.org/

NOTE

Although we properly support PGP/MIME I've discovered some systems don't, including Mailvelope: https://github.com/roundcube/roundcubemail/issues/8417#issuecomment-1040307808

Also there's the crypto refresh https://datatracker.ietf.org/doc/draft-ietf-openpgp-crypto-refresh/ And see https://fosdem.org/2024/schedule/event/fosdem-2024-2669--security-modernizing-email-encryption-the-crypto-refresh-of-openpgp/

ervee commented 3 years ago

I remember a discussion back on RainLoop Git. If I recall correctly the keys (so also your private key!) are stored server side. In my case, I'm the server admin so I kind of trust this. But I would not trust anyone else with my private key. Even if I used my own server, I would not trust it if it was a VPS or some other shared service.

So if this is currently broken, I would consider this a good candidate to shrink the SnappyMail code :)

the-djmaze commented 3 years ago

If I recall correctly the keys (so also your private key!) are stored server side.

No, private keys are in your browser localStorage (which also sucks). An "encrypted" copy could be stored on the server (with symmetric encryption and passphrase) but that defeats the use of serverside gnupg.

RoundCube: uses gnupg ProtonMail: does not use PGP (they say they do, but they send an e-mail with a link and you must open the link) etc.

There are only 2 cases where we need private keys:

  1. Sign a message
  2. Decrypt received message

Public keys is no issue and we can still use for:

  1. Verify received signature
  2. Encrypt message to send

Solving the public keys issue would be the first step to solve. This should be easy for webmail without security issues.

GregThib commented 2 years ago

ProtonMail: does not use PGP (they say they do, but they send an e-mail with a link and you must open the link) etc.

Plaît-il ? Some people send messages to me from protonmail, and I received them PGP-encrypted (as I have WKD configured in my domain, and then my public keys are easily discoverable).

Maybe you receive the message in this form when the user force encryption to a recipient who don't have a public key known or discoverable via WKD? Cause, with protonmail we are able to encrypt to recipient whithout PGP-enabled encryption, with something like a password.

GregThib commented 2 years ago

Just my two cents: in the RainLoop issue feed, there is a lot of questions about PGP implementation, e.g. the use of WKD to discover keys, the opportunistic encryption, and a few things more.

Today, I think it's important to reach a core-base PGP encryption (encrypt, sign, verify) without getting too much pressure about the remaining (trust model, key discover, etc.).

As we can see with Thunderbird, since they have dropped the Enigmail support, their implementation is suffering too.

the-djmaze commented 2 years ago

@GregThib oh yes, i've added OpenPGP.js 5.0.1 for comparison and development. Problem is that the library is not 100% compiled for web, it also (still) has Node.js code. So i'm cleaning up the 1.5 MiB code to get more in line with the old 636 KiB openpgp-sm.js

But still, i should look at GnuPG as well

the-djmaze commented 2 years ago

Working on this issue revealed that RainLoop never correctly verified signed messages. A signature must be done on the whole multipart/signed but it never does. It only checks if the pgp sgnature is valid.

GregThib commented 2 years ago

Oo This is a pretty serious issue.

the-djmaze commented 2 years ago

Oo This is a pretty serious issue.

Yep, but as you can see all the commits, there is work in progress :)

For now it uses GnuPG to verify PGP messages. This method is tested and now works properly here.

  1. uses php pecl gnupg
  2. store public keys in: …/_data_/_default_/storage/[example.com]/[account]/.gnupg (no UI yet)
  3. JavaScript click on "🔒OpenPGP signed message (click to verify)" will call the new MessagePgpVerify (result not in UI yet)

Still todo regarding "verify":

  1. Improve UI /#/settings/openpgp with public keys section for verifying received messages
  2. Show verify results in UI
  3. Alternatives to GnuPG extension: Crypt_GPG, OpenPGP.js, Mailvelope, openpgp-php + phpseclib

Todo after that:

  1. Encrypt using public keys
  2. Manage private keys
  3. Decrypt messages with private keys
  4. Sign messages with private keys
the-djmaze commented 2 years ago

Update: latest changes now properly load keyrings of:

Composer window is revamped and has no PopupsComposeOpenPgp. Instead the dropdown menu now has two options:

Sign

Only enabled when chosen identity (from) has a private key.

Encrypt

Only enabled when all recipients (to, cc, bcc) have a public key. Also all recipients must be either in Mailvelope or OpenPGP.js or GnuPG and can't be mixed. TODO: should be extended to lookup HKP servers and others to find the keys.

The system is still defunct and does not Sign nor Encrypt yet. First i need to get all keyring systems to work properly.

GregThib commented 2 years ago

Impressive!

I wanted to congratulate you very much for your work on this tool. I am impressed by the speed and efficiency with which you deal with this problem, and thank you for the attention given to your users.

Gratefully!

the-djmaze commented 2 years ago

@GregThib thank you!

Working on this, i noticed many problems that have to be dealt with:

  1. PHP PECL GnuPG doesn't import private keys. I need to call gpg --import --homedir=/home/snappymail/__data__/,,,,/.gnupg secret_key.asc
  2. gnupg_keyinfo() v1.5+ should have hidden param to list private keys, else only public keys are returned
  3. OpenPGP.js and Mailvelope can't sign/encrypt attachments or the whole message (only cleartext)
  4. etc.

So basically it is a PITA to sign/encrypt properly. But hopefully i get there eventually.

Looking at the Enigma source of RoundCube it seems that it also should have issues with it.

the-djmaze commented 2 years ago

Dropped support for PEAR Crypt_GPG because it is missing features. For best memory usage i want to stream data in/out the mailso imap stream with GnuPG in between. That way it is easy to encrypt/decrypt/sign/verify large messages (20MiB+). Crypt_GPG does have encryptFile but not an encryptStream.

Also it can't generate ECC keys.

So i'm building a new gpg.php class that shall solve it using gnupg and Crypt_GPG as examples.

the-djmaze commented 2 years ago

Improved detection of PGP/MIME encrypted messages. As reported at https://github.com/RainLoop/rainloop-webmail/issues/1848

It also nicely shows unencrypted attachments.

afbeelding

the-djmaze commented 2 years ago

Decryption now works using Mailvelope afbeelding

the-djmaze commented 2 years ago

I've updated the demo at https://snappymail.eu/demo/ You can follow progress there and test using the new OpenPGP, GnuPG and Mailvelope

You can send test e-mails to demo@snappymail.eu and see the result.

When using Mailvelope don't forget to edit the settings in Mailvelope -> Options -> Authorized Domains -> Add new entry afbeelding

Private key password: demo

-----BEGIN PGP PRIVATE KEY BLOCK-----

xYYEYfQDqBYJKwYBBAHaRw8BAQdAy+llquGb/U4M0kD2xyJoQ2pwlDN02C7X
D9067I8zdB3+CQMIu0SL71bzJVng2v4G53CCF+SXbvCXxV6vEIS+LapvPWov
XNdC0BYt9IhYdyumVe+eRwGbJO/r6mIY/oIHVuWnBM5FZcVHTEVm23WbjYaj
Ac0ZZGVtbyA8ZGVtb0BzbmFwcHltYWlsLmV1PsKMBBAWCgAdBQJh9AOoBAsJ
BwgDFQgKBBYAAgECGQECGwMCHgEAIQkQXzpc3AmtiuMWIQQsIj8g6irbTLaP
gdlfOlzcCa2K4/nPAP4uUrWr39wv+YKsNcLwHwOpljyu59iHOXA3halUbVCe
JwD/dY6JXCwMDgG+BmurPcJhS/S8Q6fjlN9hUi/za3acYATHiwRh9AOoEgor
BgEEAZdVAQUBAQdAvXl+RCkqtUqNVQ3Fj3bFTZjZOeNlI3ibK2eN6EjlnwcD
AQgH/gkDCND/LFApgNcK4OTn9H4weJlpeZkM4X6vpQKIH4D6LVkppyjNzMhj
/tkS+49qmxVy1KdWRunaqEIaes6huDKsxahIOnQPim7In6UMSzEeACrCeAQY
FggACQUCYfQDqAIbDAAhCRBfOlzcCa2K4xYhBCwiPyDqKttMto+B2V86XNwJ
rYrjolQBAMqdz8VkgMYjM7tinwUUTe4JjZoCsuhHPN6SpQLd/UzKAQDgRlbA
drl042/nJcdrBrQz3+wVzkaF0ehvihBf4/tfDw==
=BqC1
-----END PGP PRIVATE KEY BLOCK-----

-----BEGIN PGP PUBLIC KEY BLOCK-----

xjMEYfQDqBYJKwYBBAHaRw8BAQdAy+llquGb/U4M0kD2xyJoQ2pwlDN02C7X
D9067I8zdB3NGWRlbW8gPGRlbW9Ac25hcHB5bWFpbC5ldT7CjAQQFgoAHQUC
YfQDqAQLCQcIAxUICgQWAAIBAhkBAhsDAh4BACEJEF86XNwJrYrjFiEELCI/
IOoq20y2j4HZXzpc3AmtiuP5zwD+LlK1q9/cL/mCrDXC8B8DqZY8rufYhzlw
N4WpVG1QnicA/3WOiVwsDA4BvgZrqz3CYUv0vEOn45TfYVIv82t2nGAEzjgE
YfQDqBIKKwYBBAGXVQEFAQEHQL15fkQpKrVKjVUNxY92xU2Y2TnjZSN4mytn
jehI5Z8HAwEIB8J4BBgWCAAJBQJh9AOoAhsMACEJEF86XNwJrYrjFiEELCI/
IOoq20y2j4HZXzpc3AmtiuOiVAEAyp3PxWSAxiMzu2KfBRRN7gmNmgKy6Ec8
3pKlAt39TMoBAOBGVsB2uXTjb+clx2sGtDPf7BXORoXR6G+KEF/j+18P
=n8aj
-----END PGP PUBLIC KEY BLOCK-----
the-djmaze commented 2 years ago

Pfffft this was work: https://github.com/the-djmaze/snappymail/commit/e265a0f1c175a989e82ff2a6ce2411f9e9582a48

RainLoop modified the message HTML part in PHP on the server, and so did SnappyMail. But while supporting PGP/MIME multipart encrypted messages, the issue with parsing HTML and attachments became a problem because that is server-side.

So i've moved all code to be client-side so that decrypted messages act the same.

the-djmaze commented 2 years ago

Demo at https://snappymail.eu/demo/ is updated with latest changes.

You can encrypt using Mailvelope. afbeelding A Mailvelope button will appear when all recipients can receive encrypted data.

the-djmaze commented 2 years ago

Here's a preview release https://github.com/the-djmaze/snappymail/releases

the-djmaze commented 2 years ago

@GregThib you can start with https://github.com/the-djmaze/snappymail/releases/tag/2.12.0

luluwebmaster commented 1 year ago

Hello !

Note : My English is not very good, sorry for that.

I'm having a problem on my docker installation, based on @kouinkouin image ( #44 ).

Very simply, I can't decipher the encrypted emails.

Whether with OpenPGP or GnuPG, the decryption is done well (I have the message "Message encrypted by OpenPGP" after decryption), but the encrypted message is not replaced in the interface.

I note an interesting point, with GnuPG, when I look at the Ajax requests and I launch the decryption, the response contains the decrypted message, but the JSON does not seem correct ( See the screenshots ).

Note that I don't know if this is a problem specific to the Docker image, or if the problem also exists in the context of a classic installation.

Is this issue known / related to this issue?

Thanks in advance !

After click on "Decrypt" button :

image

Reponse of request when use GnuPG ( Response is decrypted, I can read message ) :

image

image

the-djmaze commented 1 year ago

@luluwebmaster Who encrypted the message? (Hopefully Thunderbird, because that has bugs and should only be used in PGP/MIME mode).

Looking at it, it is quoted-printable encoded HTML and that is wrong.

There are 2 types of PGP encryption:

  1. PGP/Inline which is plain text (not HTML)
  2. PGP/MIME which can be plain and/or HTML with optional attachments and/or inline images
luluwebmaster commented 1 year ago

The message received has been encrypted by Thunderbird.

Could this be a problem with Thunderbird ?

the-djmaze commented 1 year ago

Correct! https://superuser.com/questions/794959/what-is-introducing-quoted-printables-into-my-pgp-encrypted-emails

Force Thunderbird to use PGP/MIME and try again

luluwebmaster commented 1 year ago

Does that mean that all my correspondents must activate this option .. ? It's not possible to "fix" this on Snappy interface ?

the-djmaze commented 1 year ago

If we make a workaround, how would other people with different applications see the broken Thunderbird?

the-djmaze commented 1 year ago

Looking at it, the issue is different these days and i will look into it.

afbeelding

luluwebmaster commented 1 year ago

Ok, not problem, I wait your feedback !

luluwebmaster commented 1 year ago

Hello,

After update of docker image ( 2.26.1 ), now that works correctly !

Thank you.

LeifAndersen commented 1 year ago

@the-djmaze FWIW, I seem to have the same problem that @luluwebmaster was describing. But the message was encrypted using mailvalope instead of thunderbird.