search

signalapp / Signal-Android

A private messenger for Android.
https://signal.org
GNU Affero General Public License v3.0
25.54k stars 6.12k forks source link

Signal crashes after TB restore on another device #7400

Closed johanw666 closed 6 years ago

johanw666 commented 6 years ago

Because https://github.com/signalapp/Signal-Android/issues/7386 is closed and this is a separate issue I'm opening a new issue.

I have:

Bug description

After a succesfull upgrade to 4.16.4 up to commit https://github.com/signalapp/Signal-Android/commit/0bbe83f8f256219985c61ab2d41ad5a1df5001f3 , I tried to restore a Titanium Backup on a test device. However, Signal on that device crashes as soon as it starts.

Steps to reproduce

Actual result: When Signal is started on the other phone it crashes at startup even before the password can be entered. Expected result: Signal should show all messages that it shows on the first device.

Perhaps I miss something about SQLCipher, do I need to change some global key on the device or something like that?

Device info

Both devices are of the same type: Device: Sony Xperia Z3 compact Android version: 6.0.1 Signal version: 4.16.4

Link to debug log

Debug log is not possible to create from Signal but this is the relevant logcat output:

02-06 13:13:24.216 W/PartProvider(23183): onCreate()
02-06 13:13:24.433 D/AndroidRuntime(23183): Shutting down VM
02-06 13:13:24.435 E/AndroidRuntime(23183): FATAL EXCEPTION: main
02-06 13:13:24.435 E/AndroidRuntime(23183): Process: org.thoughtcrime.securesms, PID: 23183
02-06 13:13:24.435 E/AndroidRuntime(23183): java.lang.AssertionError: javax.crypto.AEADBadTagException
02-06 13:13:24.435 E/AndroidRuntime(23183):     at org.thoughtcrime.securesms.crypto.KeyStoreHelper.unseal(KeyStoreHelper.java:73)
02-06 13:13:24.435 E/AndroidRuntime(23183):     at org.thoughtcrime.securesms.crypto.DatabaseSecretProvider.getEncryptedDatabaseSecret(DatabaseSecretProvider.java:58)
02-06 13:13:24.435 E/AndroidRuntime(23183):     at org.thoughtcrime.securesms.crypto.DatabaseSecretProvider.getOrCreateDatabaseSecret(DatabaseSecretProvider.java:29)
02-06 13:13:24.435 E/AndroidRuntime(23183):     at org.thoughtcrime.securesms.database.DatabaseFactory.<init>(DatabaseFactory.java:121)
02-06 13:13:24.435 E/AndroidRuntime(23183):     at org.thoughtcrime.securesms.database.DatabaseFactory.getInstance(DatabaseFactory.java:60)
02-06 13:13:24.435 E/AndroidRuntime(23183):     at org.thoughtcrime.securesms.database.DatabaseFactory.getSmsDatabase(DatabaseFactory.java:75)
02-06 13:13:24.435 E/AndroidRuntime(23183):     at org.thoughtcrime.securesms.service.ExpiringMessageManager.<init>(ExpiringMessageManager.java:29)
02-06 13:13:24.435 E/AndroidRuntime(23183):     at org.thoughtcrime.securesms.ApplicationContext.initializeExpiringMessageManager(ApplicationContext.java:151)
02-06 13:13:24.435 E/AndroidRuntime(23183):     at org.thoughtcrime.securesms.ApplicationContext.onCreate(ApplicationContext.java:86)
02-06 13:13:24.435 E/AndroidRuntime(23183):     at android.app.Instrumentation.callApplicationOnCreate(Instrumentation.java:1014)
02-06 13:13:24.435 E/AndroidRuntime(23183):     at android.app.ActivityThread.handleBindApplication(ActivityThread.java:4806)
02-06 13:13:24.435 E/AndroidRuntime(23183):     at de.robv.android.xposed.XposedBridge.invokeOriginalMethodNative(Native Method)
02-06 13:13:24.435 E/AndroidRuntime(23183):     at de.robv.android.xposed.XposedBridge.handleHookedMethod(XposedBridge.java:360)
02-06 13:13:24.435 E/AndroidRuntime(23183):     at android.app.ActivityThread.handleBindApplication(<Xposed>)
02-06 13:13:24.435 E/AndroidRuntime(23183):     at android.app.ActivityThread.access$1600(ActivityThread.java:154)
02-06 13:13:24.435 E/AndroidRuntime(23183):     at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1452)
02-06 13:13:24.435 E/AndroidRuntime(23183):     at android.os.Handler.dispatchMessage(Handler.java:102)
02-06 13:13:24.435 E/AndroidRuntime(23183):     at android.os.Looper.loop(Looper.java:234)
02-06 13:13:24.435 E/AndroidRuntime(23183):     at android.app.ActivityThread.main(ActivityThread.java:5526)
02-06 13:13:24.435 E/AndroidRuntime(23183):     at java.lang.reflect.Method.invoke(Native Method)
02-06 13:13:24.435 E/AndroidRuntime(23183):     at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:726)
02-06 13:13:24.435 E/AndroidRuntime(23183):     at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:616)
02-06 13:13:24.435 E/AndroidRuntime(23183):     at de.robv.android.xposed.XposedBridge.main(XposedBridge.java:107)
02-06 13:13:24.435 E/AndroidRuntime(23183): Caused by: javax.crypto.AEADBadTagException
02-06 13:13:24.435 E/AndroidRuntime(23183):     at android.security.keystore.AndroidKeyStoreCipherSpiBase.engineDoFinal(AndroidKeyStoreCipherSpiBase.java:484)
02-06 13:13:24.435 E/AndroidRuntime(23183):     at javax.crypto.Cipher.doFinal(Cipher.java:1502)
02-06 13:13:24.435 E/AndroidRuntime(23183):     at org.thoughtcrime.securesms.crypto.KeyStoreHelper.unseal(KeyStoreHelper.java:71)
02-06 13:13:24.435 E/AndroidRuntime(23183):     ... 22 more
02-06 13:13:24.435 E/AndroidRuntime(23183): Caused by: android.security.KeyStoreException: Signature/MAC verification failed
02-06 13:13:24.435 E/AndroidRuntime(23183):     at android.security.KeyStore.getKeyStoreException(KeyStore.java:636)
02-06 13:13:24.435 E/AndroidRuntime(23183):     at android.security.keystore.KeyStoreCryptoOperationChunkedStreamer.doFinal(KeyStoreCryptoOperationChunkedStreamer.java:224)
02-06 13:13:24.435 E/AndroidRuntime(23183):     at android.security.keystore.AndroidKeyStoreAuthenticatedAESCipherSpi$BufferAllOutputUntilDoFinalStreamer.doFinal(AndroidKeyStoreAuthenticatedAESCipherSpi.java:373)
02-06 13:13:24.435 E/AndroidRuntime(23183):     at android.security.keystore.AndroidKeyStoreCipherSpiBase.engineDoFinal(AndroidKeyStoreCipherSpiBase.java:473)
02-06 13:13:24.435 E/AndroidRuntime(23183):     ... 24 more
moxie0 commented 6 years ago

Correct, TB will never work again. The key material is on hardware on the origin device, so TB will continue to work there so long as you don't uninstall the app, but uninstalling the app will clear the key material.

johanw666 commented 6 years ago

OK, so I'll have to see if it is possible to export and re-import the key material with an encrypted backup function. I'll look into that.

xyzzyxw commented 6 years ago

That's awful. The only way to backup is gone? I'm considering to stop using Signal altogether.

minnmann commented 6 years ago

As moxie mentioned here https://whispersystems.discoursehosting.net/t/signal-android-alpha-channel-for-the-brave/1909, one of the reasons to migrate the database is to enable a (built-in) backup functionality:

This release migrates data from the SQLite + ciphertext blobs storage strategy to a SQLCipher + KeyStore storage strategy. [...] This will open the door to full text message content search, as well as backup.

For now, the option to backup via TB might be gone, but in the long run this is definitely the way to go.

johanw666 commented 6 years ago

That functionality was possible all along before this change by just copying all files under /data/data/org.thoughtcrime.smssecure, just like TB does it. Now I think it can still be done but then from a modified Signal version that also copies the key from the hardware store (other apps apparently have no access to that).

While a built-in backup might eventually appear we must still wait and see if it also copies the identity keys so it will be completely seamless.

johanw666 commented 6 years ago

@xyzzyxw I did get complete backup working within Signal - I just backup and restore the database key too. But you need to do that from within the app See the discussion on the community forum at https://whispersystems.discoursehosting.net/t/titanium-backup-will-no-longer-work-starting-with-signal-4-16/1940/6?u=johanw666

sdettmer commented 5 years ago

I used Signal in the past and even tested a backup once. Now I understand that by the time the backup was intentionally silently broken by the developer. Now I am supposed to lose all my messages and generate a new key "secured" by SMS (which renders the whole tool useless, I think). I'm thinking about this now quite a while but I have no idea how the user experience could be made worse and how to lose trust more. Breaking a backup is so bad, I don't have words...

minnmann commented 5 years ago

@sdettmer I don't quite understand what you are referring to when talking about "intentionally broken backup by the developers". The current encrypted full-backup solution is working seamlessly (admittedly after some initial issues, especially in connection with SD cards) and is exporting/importing literally everything (messages, media, identity key, verification status of contacts, ...) - compared to the half-baked plaintext backup before. I have used it several times on stock Android devices (as have several friends) without any issues (both restorting on the same device as well as restoring on a new device). If you restore the backup, everything will just be exactly the same as before. On 3rd party ROMs, there are occasionally some problems. But if the ROM has a flawed implementation of Android's keystore (which is AFAIR true for several versions of LineageOS), I wouldn't call it Signal's fault.

Besides that, there are at least two community developed tools to decrypt the backup on a PC (https://github.com/xeals/signal-back, https://github.com/pajowu/signal-backup-decode).

Sure, TB backups are broken, but they have never been officially supported in the first place. And plaintext backups can no longer be imported - but they can be read anyway.

sdettmer commented 5 years ago

@minnmann The "full-backup solution" does not backup the keys at all, it only backups the chats, right? Also, I like to be free to chose my backup solution and especially this is not an own one for each app. So in total, there is no possiblity to backup at all. If your device breaks, your key is lost. You have to meet each peer again to check the new key. According to the message "The key material is on hardware on the origin device" it is intentionally made impossible. It was different, so for people like me who even test their backups, this is especially mean. Otherwise I probably never ever had started using Signal. OT: Now I have to talk to all my Signal friends explaining them why I like to switch. Interestingly, they have no problem with this "secure number has changed " message! They just click it away, because they are used the people need to change devices and that no backup is possible! They just click away the security! What a pitty.

minnmann commented 5 years ago

@sdettmer Sorry for being not clear enough in my explanation - I think there is some serious misunderstanding/misinformation: Everything - including the keys for each contact - is backed up and restored. Otherwise it wouldn't be possible (or at least make no sense) to backup/restore the verification state. E.g. my girlfriend (we verified our safety numbers long time ago) got a new phone and restored the backup of Signal on the new phone. If she wouldn't have told me that she got a new phone, I wouldn't even have noticed (no change in safety numbers and no change in verification status whatsoever).

The "key in hardware" - AFAIK - is not the keypair that is used to encrypt messages, but the key to access the SQLCipher database, where everything (messages, media, settings, keys of your contacts etc.) is stored - effectively preventing every other app to access it (while enabling features like full text search). The key being "in hardware" means that only the Signal app itself can access it (at least on stock Android; not talking about rooting and using other stuff) and this will get deleted upon uninstall.

When doing a backup, Signal uses the key "in hardware" (by accessing Android's Keystore) to open the database, copies the complete content to a backup file, encrypts this backup file with the (random) password that is shown when enabling backups and stores this in the phone's memory.

Edit - forgot to mention: In my understanding (following the discussion about a full backup solution), it has always been the explicit aim of the backup (besides media etc.) to include the keys - exactly to prevent (unwanted) key changes from happening when you switch devices or just reinstall Signal.

Having said that, this discussion should probably be continued - e.g. if anything remains unclear (I am by no means any expert on Android) or you have further questions or concerns - at the community forum: https://community.signalusers.org/.

sdettmer commented 5 years ago

@minnmann Thank you for your explanation. It is very interesting that your GFs backup worked and you did not notice, because then the communication keys apparently are not stored in a non-recoverable way, which I assumed was the motivation to change it at all. I think it could be that this hardware key is used to encrypt the data, which includes the communication key, and that a decryption error is not caught by the signal application causing a crash. Stacktrace suggests this. Thank you for your detailed explanation.

In meantime I restored a backup from 2017, which was working and got me the main key back, but this was not the best idea, because I got a lot of "errnously encrypted message" and "Message had been encrypted for a non-existing session" and I'm afraid these are lost. I should have restored the old backup in air plane mode, starting once to migrate data, then overwriting with the new, the going online. But now it is too late. Also I got several "+1 xxx xxx xxx now is using signal! The sender is not in your contact list." and one conversation seems to crash Signal when I try to open it.

BTW, how are you doing the backup practically? Manually clicking every day? An additioal problem could be that when next time the format changes and you cannot reimport an old backup in a newer Signal version, you seem have to have a recent backup. Maybe I try this, maybe I delete my Signal user account (I hope this is possible at all). I thought I do better backups than almost anyone on Android but unfortunately failed, I'm not sure how much time I should spent to get it fully working.

Maybe I create a signal built-in backup, reinstall signal and restore its backup in the hope that it fully works afterwards.

Thank you for your time and your detailed and helpful message!

minnmann commented 5 years ago

@sdettmer I am happy to be of any help! :)

BTW, how are you doing the backup practically? Manually clicking every day?

Once backups are enabled, the backup scheduler will automatically create an update every day, retaining at most two backups. Of course, storing the backups in a different place, e.g. for the case of a lost phone, is still up to you.

An additioal problem could be that when next time the format changes and you cannot reimport an old backup in a newer Signal version, you seem have to have a recent backup.

So far - only speaking for the built-in full backup feature -, the developers always made sure that you could import a backup made with an older version of Signal. E.g. this has been the case several times whenever new data was introduced to the SQLCipher database and a migration had to be done.

Maybe I try this, maybe I delete my Signal user account (I hope this is possible at all).

I would strongly recommend to give the "new" built-in backup a try. :) It is really convenient and I have never had any problems with it - after restore everything is literally the same as before. If you still want to delete your account, this is possible here: https://signal.org/textsecure/unregister/ As an alternative, going to advanced settings and unticking "Signal messages" should be of the same result.

sdettmer commented 5 years ago

This sounds great! Maybe I just had bad luck. You convinced me to try it, thank you for your time and your detailed help! Very appreciated.