Encrypted backups do not work

[mase76]

Hi! I use oandbackup with openkeychain to encrypt the backups. The gpg files are created, but restoring does not work. I cannot even open them with openkeychain directly. I set up the email of my pgp user in oandbackup. When I look into the app settings in openkeychain, I see oandbackup seems to use a deprecated api. Maybe that is the issue.

[jensstein]

ah, you're right. well, the cryptography code in oab hasn't really been changed since it was included so i don't think the bug is there. the api used for communicating with the pgp app is openpgp-api. they only have one release (v5) which is the commit that the submodule in the oab source links to. so the api should be as up to date as it can be. unfortunately they seem to have removed ant support sometime after the release so i can't test the most current code right now. and as far as i can tell from their readme they haven't changed the way apps are supposed to call the pgp app. so the code in Crypto.java should still be valid. but i also have both the issue with non-working encryptions and the "old api" message. i suspect the bug is on the openkeychain side. they might have broken some backwards compability. could you perhaps report it with them?

also, now after clearing openkeychain's data when i try to import my public key as prompted during a backup, openkeychain crashes with this exception:

E/AndroidRuntime(23874): FATAL EXCEPTION: main
E/AndroidRuntime(23874): Process: org.sufficientlysecure.keychain, PID: 23874
E/AndroidRuntime(23874): java.lang.RuntimeException: Failure delivering result ResultInfo{who=null, request=100484, result=-1, data=Intent { (has extras) }} to activity {org.sufficientlysecure.keychain/org.sufficientlysecure.keychain.remote.ui.RemoteServiceActivity}: java.lang.ClassCastException: org.sufficientlysecure.keychain.operations.results.ImportKeyResult cannot be cast to org.sufficientlysecure.keychain.operations.results.EditKeyResult
E/AndroidRuntime(23874):    at android.app.ActivityThread.deliverResults(ActivityThread.java:3351)
E/AndroidRuntime(23874):    at android.app.ActivityThread.handleSendResult(ActivityThread.java:3394)
E/AndroidRuntime(23874):    at android.app.ActivityThread.access$1300(ActivityThread.java:135)
E/AndroidRuntime(23874):    at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1244)
E/AndroidRuntime(23874):    at android.os.Handler.dispatchMessage(Handler.java:102)
E/AndroidRuntime(23874):    at android.os.Looper.loop(Looper.java:136)
E/AndroidRuntime(23874):    at android.app.ActivityThread.main(ActivityThread.java:5001)
E/AndroidRuntime(23874):    at java.lang.reflect.Method.invokeNative(Native Method)
E/AndroidRuntime(23874):    at java.lang.reflect.Method.invoke(Method.java:515)
E/AndroidRuntime(23874):    at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:785)
E/AndroidRuntime(23874):    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:601)
E/AndroidRuntime(23874):    at dalvik.system.NativeStart.main(Native Method)
E/AndroidRuntime(23874): Caused by: java.lang.ClassCastException: org.sufficientlysecure.keychain.operations.results.ImportKeyResult cannot be cast to org.sufficientlysecure.keychain.operations.results.EditKeyResult
E/AndroidRuntime(23874):    at org.sufficientlysecure.keychain.remote.ui.AccountSettingsFragment.onActivityResult(AccountSettingsFragment.java:109)
E/AndroidRuntime(23874):    at android.support.v4.app.FragmentActivity.onActivityResult(FragmentActivity.java:165)
E/AndroidRuntime(23874):    at android.app.Activity.dispatchActivityResult(Activity.java:5475)
E/AndroidRuntime(23874):    at android.app.ActivityThread.deliverResults(ActivityThread.java:3347)
E/AndroidRuntime(23874):    ... 11 more
W/ActivityManager(  515):   Force finishing activity org.sufficientlysecure.keychain/.remote.ui.RemoteServiceActivity

[john156]

:+1:

[DJaeger]

I have also seen an issue with restoring encrypted backup last days. oab gave me an error prompt while restoring. But I have determined after several tries, that the app seems to be successfully restored.

[mase76]

Some of the apps restore, but only a little few. Most don't. With disabled encryption most restores work, except some special cases like openvpn for android. But that is an other issue I reported in the past.

[DJaeger]

I have only used the restore funtion for firefox in the last days, as it again lost my complete session. So I started restore from oab, it was working very long (also the time before it asked for my pgp key's password was long) and than it gave me an error message, something with crypto. But my firefox settings and session were all restored.

[jensstein]

that sounds strange. but still, openkeychain produces files which cannot be decrypted with gnupg so something is basically broken. and since oab hasn't really changed anything there and is using the most current release of the api library i would guess it's a bug with openkeychain. if anybody reports it there or finds something useful in their bug tracker, please link it here also. has anybody tried any other pgp apps (i think there's at least one other working with the openpgp-api library)? and does anybody have an old version of openkeychain to check against?

@DJaeger try looking in your oandbackup.log and see if the error messages contain anything useful.

[dschuermann]

@jensstein Sry, we forgot to tag newer version of openpgp-api. I now tagged v8. It is now also available via jcenter, see https://github.com/open-keychain/openpgp-api#1-add-the-api-library-to-your-project which is now the preferred method to include the library. The problem could be related to not closing the input/outputs properly. We fixed this in https://github.com/open-keychain/openpgp-api/commit/a5a3e2b3014cba2f1a7d356731fc3e3d066c6c62

Maybe you need to adapt the code by comparing it to our example at https://github.com/open-keychain/openpgp-api/tree/master/example

Unfortunately, because we are only a small dev team, we don't have much time currently to help other developers improve their apps. But if you have someone from your side I can advise him/her how to improve the usage of our API. One obvious improvement: In your settings, instead of using a list of openpgp user ids, you could use ACTION_GET_KEY_IDS like in https://github.com/open-keychain/openpgp-api/blob/master/example/src/main/java/org/openintents/openpgp/example/OpenPgpApiActivity.java#L388 . Without specifying the EXTRA_USER_IDS it will allow you to select from all public keys.

[dschuermann]

BTW: A list of changes is here: https://github.com/open-keychain/openpgp-api/blob/master/openpgp-api/src/main/java/org/openintents/openpgp/util/OpenPgpApi.java#L40

[jensstein]

this should now be fixed with 72d69fd73801aa484492aa6e5d50532a01d9e1f2. i will add instructions for building the latest version of the library with ant later but if anybody wants to test it out without building there's an apk here: https://drive.google.com/file/d/0B0bNUNWd2wPPQUhCN3lSTGJLLTQ/view?usp=sharing

@dschuermann it would be nice if you cared a little more about backwards compability and transparency. maintaining a proper changelog and notifying of changes to e.g. build system or class names would be helpful (i found out with git log that OpenPgpListPreference which i used had been renamed).

@all i'm thinking about making this library optional and accessing it's classes through reflection, in case anybody is interested.

[ildar]

Is it the way K9 uses APG for encryption?

[dschuermann]

@jensstein Thanks for your feedback. I improved the README and moved the CHANGELOG into an extra file in https://github.com/open-keychain/openpgp-api

[jensstein]

@ildar i'm not familiar with k9 mail but their gradle build does include openpgp-api-library as a dependency so i would imagine they are doing something similar to what i'm doing here. i don't know if they treat apg differently.

@dschuermann great. this already looks more accessible with the separate changelog. you could also consider making deprecated and changed stuff issue warnings for a period instead of making the switch or deleting the class between to versions (i'm thinking specifically of OpenPgpListPreference -> OpenPgpAppPrefence).

[mase76]

I hope, it arrives in f-droid, soon.

[jensstein]

@mase76 well, realistically it probably won't be too soon. i'd like to have some confirmations that the issue is fixed. and i also have to write up some build instructions. aside from the other issues that should probably be fixed for a new version. so i'd encourage you to check out the apk i posted.

[dschuermann]

@jensstein out of curiosity: Why are you using ant and don't switch to gradle? I'd like to understand the issues here.

[mase76]

On my device with cm12.1 the app crashes when trying to enter the encryption settings menu. I installed your test build.

[jensstein]

@mase76 could you please send a logcat to my email? otherwise it's a little hard to tell what's wrong.

[jensstein]

@dschuermann one of the reasons for using ant instead of gradle is that i really dislike the way gradle works by self-installing through a shell script. and i don't really like the automatic and almost silent dependency management. i want to go out and get the dependencies myself (when building something from source) or at least be explicitly asked for confirmation before download. gradle didn't do that the last time i used it - but of course i didn't look into it's settings either, so i can have missed something. another important factor is that the android build system gets more and more ide-centric. and since i just use a plain text editor and the command line for everything, the official instructions and things like stackoverflow get less and less useful for me. but ant works perfectly well in that workflow. but if you feel there's something i have missed about gradle or if i have any misconceptions, please feel free to tell me.

[mase76]

@jensstein Did you get the logcat I have sent?

[jensstein]

@mase76 no, i'm afraid i didn't. did you send it to j.stn.oab@gmail.com? coincidentally i just made another test apk when i added some new translations and i think the bug may be fixed there: https://drive.google.com/file/d/0B0bNUNWd2wPPMDY0LVVTd3Exdlk/view?usp=sharing please check it out :)

[mase76]

Yes, I sent it to this address. I tested your new build. The crash is fixed. Encryption seems to work now. I backed up and restored some apps without problems. I think, you can close this issue now.

[jensstein]

that's weird, i just checked that account again just now but there was nothing from you. but great that the issues are resolved. let's keep the issue open at little while yet (at least until i write some proper build instructions for the new library version) for visibility.

[dschuermann]

@jensstein Regarding gradle: a) No need to use the gradle-wrapper (the self-installing shell script). The F-Droid devs also avoid its use. b) Regarding dependency mgnt: In comparison to ant, it supports real dependencies. But yeah you are right that it works in the background without much configuration. You can always use "--offline" to prevent gradle from downloading stuff and you must not include any Maven repositories. Then only local resources are used. c) IDE: Gradle is totally independent from the IDE. Actually when I used ant+Eclipse, Eclipse actually didn't use the ant build files. My current setup is gradle+Android Studio (AS), and AS actually uses the gradle build system in the background. I still build release versions from command line only with "gradle clean; gradle build".

[jensstein]

@dschuermann thanks for the explanation :) i still don't like it though (i still think my points stand) so i think i'll stick with ant for the forseeable future. regarding ides i just mean that the change of (officially endorsed) build system has happened alongside a shift towards almost exclusive focus on ides, for example in the documentation. that's my perception at least. this is just to explain why i feel estranged enough from the android "community" to refrain from following that trend. i do know that gradle is absolutely independent from any ide. i have built a number of the android programs i use from source using gradle (including openkeychain at some point) so i'm not unfamiliar with how it works. of course, if anybody reports any issues with using ant, i'm open to listening.

[DJaeger]

Your test apk is forcing encryption. It tries to decrypt my not encrypted backups with OpenKeychain also if I do not have encryption enabled and no encryption provider chosen. In addition the decryption also starts if I cancel the screen of OpenKeychain to choose the key to use.

[DJaeger]

It looks some more strange. It seems it has tried to encrypt the backup I wanted to restore, and now its broken. Let me check that some more deeper.

[dschuermann]

In some rare cases the problem could be caused due to https://github.com/open-keychain/open-keychain/issues/1504 . This will be fixed in the next OpenKeychain version.

[jensstein]

@DJaeger did you only see the problem with one app? it would be really helpful if you could find out if it's reproduceable (i can't test it myself right now).

[dschuermann]

@jensstein We have a new API version out: v9 on https://github.com/open-keychain/openpgp-api . It fixes a problem where data streams could end up truncated, so I advise to upgrade.

[cannycartographer]

Hi, does Oandbackup support encryption/decryption with apg? I had that installed but every time I backed up encryption failed with an error (so I just got an unencrypted backup). oandbackup doesn't seem to recognise apg in the providers list.

I'm using the 0.2.11 version, f-droid build.

[cannycartographer]

And should I expect encryption and decryption to work with openkeychain (which I've now installed) with that version?

Thanks!

[jensstein]

encryption is implemented using the openpgp-api library so any app using that api should be supported. however, they have explicitly blacklisted apg in OpenPgpAppPreference.java with the comment: "Unfortunately, the current released version of APG includes a broken version of the API". i don't know which version of apg they refer to and i don't know if apg has since resolved the issue on their side. when no provider is chosen, oab defaults to openkeychain so that could probably be the cause of your error if it wasn't installed.

yes, you should expect encryption to work with openkeychain. 0.2.11 of oab fixes an encryption bug so use that. there was an issue on the openkeychain side some releases back but i think any recent release should be fine.

[cannycartographer]

Ok, thanks a lot. Will try with openkeychain from now on.

Would it be useful for you if I sent you the error text screenshots I got (when I only had APG installed?)

[jensstein]

yes, please do if you have them handy :)

[cannycartographer]

Hope they help. I can't confirm that the errors were exactly the same on previous occasions, but I think they were similar.

[jensstein]

cool :) and please report if you have any issues encrypting with openkeychain.

[cannycartographer]

yep will do