Copy personal key into app data and import it automatically if no account found

[h-2]

I finally managed to wipe my phone and now I am able to install and upgrade applications again. But unfortunately I cannot restore my Kontalk setup. I have made a backup with oandbackup and "restored" that, but when I open Kontalk, I only see the top bar and the rest stays white. After some waiting I am then presented with the first-start-screen, i.e. where I can enter my phone number. But I would like restore my old key and more importantly my old conversations.

Have you had similar experiences? Does the on-disk format of the messages or settings somehow depend on some environment?

As always, thanks for your help and thanks for Kontalk!

[daniele-athome]

@h-2 database recovery is a delicate operation. It depends on what version the database was created and what version you're using now (I suppose is 3.0-rc2). From what version are you upgrading from? I'm asking because some upgrade from some very unstable versions (such as one beta I don't remember exactly wish) were deprecated and actually support was removed from it because it was too complicated to maintain. Of course this is going to change with future versions, but 3.0 was rapidly changing.

[h-2]

I backupped the original apk with the data and restored the same apk I was using before, beta8.. I didn't upgrade yet, because I thought it would be easier to use the same version first of all.

[daniele-athome]

Ok I think I understand what is going on here: probably the account isn't backed up (or cannot be restored) by the backup utility. So Kontalk just sees that and brings you to the registration screen, erasing everything in the process. I suggest you register again from scratch with a newly installed rc2 version, using import key function from the registration screen (you did export your key through Kontalk preferences didn't you? You should have kontalk-keys.zip into your external storage). After that, kill the app and manually restore the messages.db file into the app's database folder. I'm assuming you can use adb shell here, otherwise I can provide you a step-by-step.

[h-2]

This is the backup log from oandbackup:

{
    "label": "Kontalk",
    "versionName": "3.0-beta8.1",
    "versionCode": 42,
    "packageName": "org.kontalk",
    "sourceDir": "\/data\/app\/org.kontalk-1\/base.apk",
    "dataDir": "\/data\/data\/org.kontalk",
    "lastBackupMillis": 1429448178002,
    "isEncrypted": false,
    "isSystem": false,
    "backupMode": 3
}

There is an apk in the archive and a zip file that includes the following:

Archive:  org.kontalk.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
    14252  04-19-2015 14:56   org.kontalk/cache/com.android.opengl.shaders_cache
    40751  04-19-2015 14:56   org.kontalk/cache/imageK8n8K.png
    44847  04-19-2015 14:56   org.kontalk/cache/imager4if6.png
    67288  04-19-2015 14:56   org.kontalk/cache/imagee1jxq.png
    46466  04-19-2015 14:56   org.kontalk/cache/imageAQzNH.png
    73405  04-19-2015 14:56   org.kontalk/cache/imagepD40X.png
    63288  04-19-2015 14:56   org.kontalk/cache/imageGM4Vg.png
    92254  04-19-2015 14:56   org.kontalk/cache/imagerj8hQ.png
    98137  04-19-2015 14:56   org.kontalk/cache/image0Hrap.png
   109512  04-19-2015 14:56   org.kontalk/cache/image5RRE1.png
    96618  04-19-2015 14:56   org.kontalk/cache/imageNcMeB.png
   137268  04-19-2015 14:56   org.kontalk/cache/imagerTpdc.png
    79233  04-19-2015 14:56   org.kontalk/cache/image37Kd4.png
    64451  04-19-2015 14:56   org.kontalk/cache/imageQUoVN.png
   138386  04-19-2015 14:56   org.kontalk/cache/imaget2z73.png
    51395  04-19-2015 14:56   org.kontalk/cache/image09sSZ.png
   675840  04-19-2015 14:56   org.kontalk/databases/messages.db
   123632  04-19-2015 14:56   org.kontalk/databases/messages.db-journal
   163840  04-19-2015 14:56   org.kontalk/databases/users.db
    94904  04-19-2015 14:56   org.kontalk/databases/users.db-journal
    20480  04-19-2015 14:56   org.kontalk/databases/status.db
     8720  04-19-2015 14:56   org.kontalk/databases/status.db-journal
     1480  04-19-2015 14:56   org.kontalk/shared_prefs/org.kontalk_preferences.xml
      124  04-19-2015 14:56   org.kontalk/shared_prefs/WebViewChromiumPrefs.xml
      351  04-19-2015 14:56   org.kontalk/shared_prefs/emojicon.xml
    34816  04-19-2015 14:56   org.kontalk/app_webview/Web Data
      512  04-19-2015 14:56   org.kontalk/app_webview/Web Data-journal
  2682038  04-19-2015 14:56   org.kontalk/files/background.png
---------                     -------
  5024288                     28 files

For obvious reasons I do not want to post the files themselves ;) is ther anything saved n any other directory that oandbackup might have missed?

I have a key-backup independently of the full one, but I am more interested in my conversations...

Thanks for your help!

[h-2]

Ah, we posted at the same time. I will do as you suggested, I know my way around the shell ;)

[TheLastProject]

@h-2 For what it's worth, when using oandbackup (due to low disk space), I had my chat logs reappear after force-closing Kontalk and starting it again. May want to experiment with force-closing Kontalk (yes, ugly, but it worked for me).

[h-2]

The way daniele suggested worked for me, thanks for the quick help! Now I am experiencing the "people not in group do not appear"-issue for the first time, but thats a different issue ;)

[daniele-athome]

Guys please don't play with these things :-) once you break something it's hard to debug and analyze issues for us.

[h-2]

PS: An "export everything feature" would be great, but other things are more important now...

[daniele-athome]

@h-2 for the contacts issue, please see 88adfdeb61cedc843dcb280f873a507b5026cf82 I will probably publish rc3 tomorrow or the day after that.

[h-2]

@daniele-athome yes, I had already seen that, I am subscribed to the project :)

[h-2]

I would politely like to reopen this issue again. If you disagree, you can of course close it again.

Using backups is becoming more and more common-place, but I think Kontalk will want to specifiy explicitly not to use the google cloud for android6. However other solutions should work then. Oandbackups is the most commonly used Free Software solution and in combination with owncloud or syncthing allows very easy weekly backups of all your apps including data. Restoring Kontalk from backup doesn't work at all right now, the key is lost, the registration is lost and also all content. I am able to restore the key manually and also restore the discussions as described above, but it is quite obscure. I know the account information is not saved with the other things per se, but somehow other tools manage this, as well, e.g. Telegram justs opens after a restore and everything works as expected, including secret chat (I know Telegram sucks in a lot of other ways, but they tend to have very good usability and the source is open).

[daniele-athome]

@h-2: you're completely right. Some discussion over this can be found in #565. Telegram obviously doesn't use the platform the right way (this is sacrifying security for the sake of useness, a battle that I will always have to fight). I didn't check the source code, but something tells me that it doesn't use the Account Manager as Kontalk does.

That being said, I'm currently looking for a solution to the backup problem. Please see my latest comment on #565. If you think I've answered your question, you may close this, otherwise just reply back :-)

P.S. oandbackup doesn't seem to handle accounts at all, so I don't see a way to leverage that without doing some security compromise, such as storing the personal key in the app data directory.

[h-2]

P.S. oandbackup doesn't seem to handle accounts at all, so I don't see a way to leverage that without doing some security compromise, such as storing the personal key in the app data directory.

Why would this be a security compromise? If something can read the file it would need to run with Kontalk's privileges (or even root) -- but then it can extract the key anyway...

[daniele-athome]

I've been using AccountManager because there is a dedicated API. And because I hoped that Google came out soon enough with a keychain API that I could use to securely store the keys - which could be linked to the AccountManager. Despite the KeyChain API and all the stuff recently introduced, there still isn't an API flexible enough for Kontalk. However, I guess that the AccountManager is as much exploitable as it's the app data directory (when working as root I mean), so I guess moving the key to app data doesn't change things as much... besides, the private key itself is encrypted and the passphrase is stored in the AccountManager, so even if you use oandbackup for Kontalk, you will need to type in the passphrase at least once at the first start. I will leave this pending until I can put enough effort to see what's the impact on the source code for this.

[h-2]

Ok, thanks for the feedback, I agree with your analysis.

[daniele-athome]

Actually, moving the key to the app data would kill the account when you delete app data from Android settings (I consider this a major feature: if you delete Gmail app data, you still have your Google account in place). We could however copy (instead of move) the key outside of the AccountManager at login time. At startup time, if no account is found but a key is found in the app data directory, it will be imported as a new Kontalk account.