search

sup-heliotrope / sup

A curses threads-with-tags style email client (mailing list: supmua@googlegroups.com)
http://sup-heliotrope.github.io
GNU General Public License v2.0
900 stars 97 forks source link

Encrypt not working (was: Encrypt does not pick key via UserID, but via email only?) #474

Open schoeke opened 9 years ago

schoeke commented 9 years ago

I am not sure if I am experiencing a bug or if it is a feature request.

I want to send an encrypted email to test@example.com with the key for testing@example.com. The email address test@example.com is added to the key as a UID. This fails with:

Problem sending mail: Exception in GPGME call: General error

If I want to sign and encrypt, the error message changes:

Problem sending mail: Exception in GPGME call: Unusable public key

As said, not sure if bug or not. If it is a bug, I am not sure how I could debug it much deeper at the moment. My setup details:

My more general suggestion would be, that if encryption was requested and no public key can be found to offer to select one explicitly (by entering a key id for example).

schoeke commented 9 years ago

It seems like this issue is not related to UIDs. It seems like this happens in all encrypt steps now. Maybe related to #471? Help would be appreciated.

gauteh commented 9 years ago

could you try with an older version of gpgme?

schoeke commented 9 years ago

I deleted 2.0.10 and installed 2.0.9, but the errors persist.

schoeke commented 9 years ago

I tested gpgme down to 2.0.4 which is the lowest version in my repositories. The error persists across all versions. I also tested --use-system-libraries for 2.0.4 but no change.

gauteh commented 9 years ago

Nice work. Maybe it is a problem with the gpgme c library? Gpgme probably builds towards whatever is available

Den søndag 20. september 2015 skrev Andrej Schoeke notifications@github.com følgende:

I tested gpgme down to 2.0.4 which is the lowest version in my repositories. The error persists across all versions. I also tested --use-system-libraries for 2.0.4 but no change.

— Reply to this email directly or view it on GitHub https://github.com/sup-heliotrope/sup/issues/474#issuecomment-141783590.

schoeke commented 9 years ago

So, I think the issue arises from ruby gpgme in combination with GPGME 1.6.0. The ruby source code showed changes to "expose new API". I am not sure if this is it or not.

I can sign, I can verify, but neither can I Sign and Encrypt ("Problem sending mail: Exception in GPGME call: Unusable public key", but get asked for a passphrase) nor Encrypt ("Problem sending mail: Exception in GPGME call: General error", not getting asked for a passphrase).

I do not know how to test it further right now. #475 and #471 might be related.

Please let me know, if I can provide more information in anyway, if it helps to solve it. Sup is useless for me w/o encrypt.

gauteh commented 9 years ago

Did you try to use an older version of the GPGME C library?

Andrej Schoeke writes on September 21, 2015 16:33:

So, I think the issue arises from ruby gpgme in combination with GPGME 1.6.0. The ruby source code showed changes to "expose new API". I am not sure if this is it or not.

I can sign, I can verify, but neither can I Sign and Encrypt ("Problem sending mail: Exception in GPGME call: Unusable public key", but get asked for a passphrase) nor Encrypt ("Problem sending mail: Exception in GPGME call: General error", not getting asked for a passphrase).

I do not know how to test it further right now. #475 and #471 might be related.

Please let me know, if I can provide more information in anyway, if it helps to solve it. Sup is useless for me w/o encrypt.

Reply to this email directly or view it on GitHub: https://github.com/sup-heliotrope/sup/issues/474#issuecomment-142000307

schoeke commented 9 years ago

Unfortunately, I can't. I installed gpgme with brew and the only version I can find there is the most current.

The gpgme ruby gem in 2.0.10 exposes a new API when you look at the code, but this should not be the problem as I tested gpgme-gem 2.0.10 to 4.

gauteh commented 9 years ago

hm, ok. seems like this is the most obvious next thing to try out.

Andrej Schoeke writes on September 21, 2015 18:48:

Unfortunately, I can't. I installed gpgme with brew and the only version I can find there is the most current.

The gpgme ruby gem in 2.0.10 exposes a new API when you look at the code, but this should not be the problem as I tested gpgme-gem 2.0.10 to 4.

Reply to this email directly or view it on GitHub: https://github.com/sup-heliotrope/sup/issues/474#issuecomment-142039683

gauteh commented 9 years ago

Note that this can be tested by doing 'rake test' in the git repository (with the gpgme gem installed), as per #471.

schoeke commented 9 years ago

I ran rake test on the dev version and everything came back fine, but I can't test encrypt in this version, as it crashes for me.

gauteh commented 9 years ago

you mean check out a release branch and test it?

Andrej Schoeke writes on September 22, 2015 17:10:

I ran rake test on the dev version and everything came back fine, but I can't test encrypt in this version, as it crashes for me.

Reply to this email directly or view it on GitHub: https://github.com/sup-heliotrope/sup/issues/474#issuecomment-142319020

schoeke commented 9 years ago

I checked the release branch, where the error simply is the same and rake test does not show any problems. However, I still get the same errors when running that version. Using that branch, I got one crypto related warning: <Path to folder>/sup/lib/sup/crypto.rb:462: warning: shadowing outer local variable - subkey.

So, I thought I test the dev branch as well to see if there are any differences.

gauteh commented 9 years ago

so 'rake test' completes on release without error. but you still get an error when using your key in sup? what is that error?

release: https://travis-ci.org/sup-heliotrope/sup/builds/67330652

I restarted it.

Andrej Schoeke writes on September 23, 2015 8:51:

I checked the release branch, where the error simply is the same and rake test does not show any problems. However, I still get the same errors when running that version. So, I thought I test the dev branch as well to see if there are any differences.

Reply to this email directly or view it on GitHub: https://github.com/sup-heliotrope/sup/issues/474#issuecomment-142511820

gauteh commented 9 years ago

It seems that the tests fail because the keys have expired. Have any of your keys expired?

schoeke commented 9 years ago

No. None of my keys are expired.

Error message for encrypt: Problem sending mail: Exception in GPGME call: General error, not asking me for my passphrase.

Error message for sign and encrypt: Problem sending mail: Exception in GPGME call: Unusable public key, asking me for my passphrase. Again, this error message comes up independent of whom I am writing to.

gauteh commented 9 years ago

do the keys work if you use them manually with gnupg? the reason I am asking is that the travis output gives different/illogical errors, while one of the errors were related to an expired key.

Andrej Schoeke writes on September 23, 2015 11:41:

No. None of my keys are expired.

Error message for encrypt: Problem sending mail: Exception in GPGME call: General error, not asking me for my passphrase.

Error message for sign and encrypt: Problem sending mail: Exception in GPGME call: Unusable public key, asking me for my passphrase. Again, this error message comes up independent of whom I am writing to.

Reply to this email directly or view it on GitHub: https://github.com/sup-heliotrope/sup/issues/474#issuecomment-142547243

schoeke commented 9 years ago

The keys work "manually". I have no problems doing everything on the CLI and then copying it into a mail.

nanooq commented 9 years ago

I apologize for jumping in.

@schoeke Concerning expired keys, are you using this one: b802d6c4d25e5e87 ? You and I were testing encryption in the other thread using that key, the one also shown here: https://sks-keyservers.net/pks/lookup?op=vindex&search=schoeke . The keyserver says it is expired, which equals to what my local gpg says about it, after fetching that key. Could you please confirm on it being expired since 2015-04-02?

@gauteh How do I run a rake test locally?

schoeke commented 9 years ago

Yes, I am using this one. However, it is not expired. It expires in 2017. I am not sure why the keys on the server show a wrong date. Let me check if republishing helps.

[EDIT] It seems like the update expiration date was not properly pushed to the keyservers. Refresh and let's try again, I'd say? However, locally, it was always a valid key/not expired.

gauteh commented 9 years ago

@nanooq: type rake test in the sup git root. you need a number of dependencies for running the test, but you should be able to figure that out.

nanooq commented 9 years ago

@schoeke I just resent you an encrypted only e-mail.

this is my output of rake test:

Finished in 1.675358s, 30.4413 runs/s, 78.1923 assertions/s.

  1) Error:
Redwood::TestCryptoManager#test_sign:
Redwood::CryptoManager::Error: Exception in GPGME call: Unusable secret key
    /home/lora/Sync/sup/lib/sup/crypto.rb:143:in `rescue in sign'
    /home/lora/Sync/sup/lib/sup/crypto.rb:134:in `sign'
    /home/lora/Sync/sup/lib/sup/util.rb:610:in `method_missing'
    /home/lora/Sync/sup/test/test_crypto.rb:62:in `test_sign'

  2) Error:
Redwood::TestCryptoManager#test_encrypt:
Redwood::CryptoManager::Error: Exception in GPGME call: Invalid value
    /home/lora/Sync/sup/lib/sup/crypto.rb:182:in `rescue in encrypt'
    /home/lora/Sync/sup/lib/sup/crypto.rb:173:in `encrypt'
    /home/lora/Sync/sup/lib/sup/util.rb:610:in `method_missing'
    /home/lora/Sync/sup/test/test_crypto.rb:72:in `test_encrypt'

  3) Error:
Redwood::TestCryptoManager#test_sign_and_encrypt:
Redwood::CryptoManager::Error: Exception in GPGME call: Invalid value
    /home/lora/Sync/sup/lib/sup/crypto.rb:182:in `rescue in encrypt'
    /home/lora/Sync/sup/lib/sup/crypto.rb:173:in `encrypt'
    /home/lora/Sync/sup/lib/sup/crypto.rb:210:in `sign_and_encrypt'
    /home/lora/Sync/sup/lib/sup/util.rb:610:in `method_missing'
    /home/lora/Sync/sup/test/test_crypto.rb:80:in `test_sign_and_encrypt'

  4) Error:
Redwood::TestCryptoManager#test_verify:
Redwood::CryptoManager::Error: Exception in GPGME call: Unusable secret key
    /home/lora/Sync/sup/lib/sup/crypto.rb:143:in `rescue in sign'
    /home/lora/Sync/sup/lib/sup/crypto.rb:134:in `sign'
    (eval):1:in `sign'
    /home/lora/Sync/sup/test/test_crypto.rb:102:in `test_verify'

  5) Error:
Redwood::TestCryptoManager#test_decrypt:
Redwood::CryptoManager::Error: Exception in GPGME call: Invalid value
    /home/lora/Sync/sup/lib/sup/crypto.rb:182:in `rescue in encrypt'
    /home/lora/Sync/sup/lib/sup/crypto.rb:173:in `encrypt'
    (eval):1:in `encrypt'
    /home/lora/Sync/sup/test/test_crypto.rb:88:in `test_decrypt'

  6) Error:
Redwood::TestCryptoManager#test_verify_unknown_keytype:
ArgumentError: NULL pointer given
    /home/lora/.rvm/gems/ruby-2.1.5/gems/gpgme-2.0.10/lib/gpgme/ctx.rb:480:in `gpgme_op_sign_result'
    /home/lora/.rvm/gems/ruby-2.1.5/gems/gpgme-2.0.10/lib/gpgme/ctx.rb:480:in `sign_result'
    /home/lora/.rvm/gems/ruby-2.1.5/gems/gpgme-2.0.10/lib/gpgme/crypto.rb:263:in `rescue in block in sign'
    /home/lora/.rvm/gems/ruby-2.1.5/gems/gpgme-2.0.10/lib/gpgme/crypto.rb:260:in `block in sign'
    /home/lora/.rvm/gems/ruby-2.1.5/gems/gpgme-2.0.10/lib/gpgme/ctx.rb:79:in `new'
    /home/lora/.rvm/gems/ruby-2.1.5/gems/gpgme-2.0.10/lib/gpgme/crypto.rb:254:in `sign'
    /home/lora/Sync/sup/lib/sup/crypto.rb:140:in `sign'
    (eval):1:in `sign'
    /home/lora/Sync/sup/test/test_crypto.rb:111:in `test_verify_unknown_keytype'

51 runs, 131 assertions, 0 failures, 6 errors, 1 skips
schoeke commented 9 years ago

I could decrypt and encrypt+sign. o_O I'll check something else...

schoeke commented 9 years ago

facepalms Sup does not allow to send with a key that is not associated via UserIDs with the sending email address. This was the cause of the error. I would consider this not quite a bug but a huge annoyance as gpg does not allow for domain wide keys and I use one key for all addresses and chose this key in the config. I would need to add keys or UserIDs for each and every email address on my server (several hundred), which would also make them public as well.

I would expect sup to use the specified key in all cases or ask me at least to use another. The error messages here were not helpful. Maybe I get around to write a quick fix soon, if there is interest.

From my side, this bug can be closed.

schoeke commented 9 years ago

And I retract. While it worked once, the error occured now again even though, the sender's address and the receiver address were both in the key chain. Furthermore, it seems like that when the encryption was done, only my public key was used and not also the receivers public key.

gauteh commented 9 years ago

@nanooq: you could try to regenerate the keys, i believe they are exipred. there is a script in there.

nanooq commented 9 years ago

Ah, I see. To be honest, @schoeke had to help me out a little, too.

Voila, the new output:

lora@freedom:~/Sync/sup$ rake test
/home/lora/.rvm/rubies/ruby-2.1.5/bin/ruby -I"lib:test" -I"/home/lora/.rvm/gems/ruby-2.1.5/gems/rake-10.4.2/lib" "/home/lora/.rvm/gems/ruby-2.1.5/gems/rake-10.4.2/lib/rake/rake_test_loader.rb" "test/integration/test_label_service.rb" "test/integration/test_maildir.rb" "test/integration/test_mbox.rb" "test/test_crypto.rb" "test/test_header_parsing.rb" "test/test_helper.rb" "test/test_message.rb" "test/test_messages_dir.rb" "test/test_yaml_migration.rb" "test/test_yaml_regressions.rb" "test/unit/service/test_label_service.rb" "test/unit/test_contact.rb" "test/unit/test_horizontal_selector.rb" "test/unit/test_locale_fiddler.rb" "test/unit/test_person.rb" "test/unit/util/test_query.rb" "test/unit/util/test_string.rb" "test/unit/util/test_uri.rb" 
Some YAML tests are skipped on Ruby 2.1.0 and newer.
[2015-09-30 16:24:51 +0200] WARNING: MiniTest::Unit::TestCase is now Minitest::Test. From /home/lora/Sync/sup/test/unit/test_locale_fiddler.rb:4:in `<top (required)>'
Run options: --seed 6704

# Running:

.....{:protocol=>0, :armor=>true, :textmode=>true, :signer=>"sup-test-1@foo.bar", :sign=>true, :recipients=>["sup-test-2@foo.bar", "sup-test-1@foo.bar"]}
.{:protocol=>0, :armor=>true, :textmode=>true, :recipients=>["sup-test-2@foo.bar", "sup-test-1@foo.bar"]}
...{:protocol=>0, :armor=>true, :textmode=>true, :recipients=>["sup-test-2@foo.bar", "sup-test-1@foo.bar"]}
.......[2015-09-30 16:24:52 +0200] WARNING: problem reading message sup-faked-df001467cbd7d1c987812838111e7731
.[2015-09-30 16:24:52 +0200] WARNING: problem reading message sup-faked-b4f27f6a974091c7c1c67484b827eb61
......[2015-09-30 16:24:52 +0200] WARNING: found invalid date in potential mbox split line, not splitting: "From sea to shining sea\n"
[2015-09-30 16:24:52 +0200] WARNING: found invalid date in potential mbox split line, not splitting: "From bob@bob.com I get only spam.\n"
....................S.......

Finished in 2.156099s, 23.6538 runs/s, 69.5701 assertions/s.

51 runs, 150 assertions, 0 failures, 0 errors, 1 skips

You have skipped tests. Run with --verbose for details.
schoeke commented 9 years ago

We, @aschoeke and @nanooq , found that we are facing two different problems in this single issue. @gauteh, we will lay them out to you and anyone in this thread, knowing that splitting them up might be an option.

Andrej: While I implemented the always trust option explained in https://github.com/sup-heliotrope/sup/wiki/gpg this was not in effect apparently. Manually trusting the keys works. However, a further problem is, that sup only uses the default key if the sender's address matches the keys primary UID exactly. This is bad design as choosing a default key for encryption in the config implies that this key is always used. Furthermore, there is no feedback to this effect. There is no way currently to manually choose a key as far as I can see.

Nanooq: The rake test finally ran through as expected, well, at least it did not show the crypto-part to be broken. I was successful in sending @schoeke a plain email, followed by a signed email, followed by an encrypted email. Sending an signed+encrypted email fails, though: After failing three passphrase attemps sup displays: Problem sending mail: Exception in GPGME call: Bad passphrase Manually encrypting and decrypting in gpg works and I am pretty certain that I do not need more than three attempts for the correct passphrase. I was poking around in crypto.rb a little. These are the parameters passed to gpgme: :protocol=>0, :armor=>true, :textmode=>true, :signer=>"A0B32504", :sign=>true, :always_trust=>true, :recipients=>["keybase.io@andrej-schoeke.de", "nanooq@nanooq.org"]} A0B32504 denoted the correct key, the email addresses are also correct.

vcarluer commented 7 years ago

Your encryption key may not be trust enough. I had the problem because I reimported my key from files. I solved this issue by doing this: http://stackoverflow.com/questions/9460140/gpg-encrypt-file-without-keyboard-interaction

$ gpg2 --edit-key {recipient email address}

trust 5 (select 5 if you ultimately trust the key) save