Import/Export of Sites

[djrieger]

I'd like to be able to import sites I exported using the Mac app from masterpasswordapp.com. The Mac app exports (site name, password type, counter) to an .mpsites file. Exporting the sites from the Android app would also be great.

Related to #1

[dkunzler]

I don't have a mac. Can you provide me with such a file. You can of course export garbage data. Without I don't see a possibility to do it. And even with a file I have to think about it because I don't know how static the structure of this file ist. This would be another place (apart from the algorithm) where I have to maintain compatibility.

[djrieger]

This is what such a file looks like:

# Master Password site export
#     Export of site names and stored passwords (unless device-private) encrypted with the master key.
# 
##
# Version: 20107129812017
# User Name: John Doe
# Key ID: c1537dXXXXXXXXXXXXXXXXXXXXXXXXXXXd03a02e1
# Date: 2014-10-01T19:19:03Z
# Passwords: PROTECTED
##
#
#               Last     Times  Password                  Site  Site
#               used      used      type                  name  password
2014-10-01T18:39:09Z         1      17:1                 site1  
2014-09-03T18:50:51Z         5      16:1                 site2  
2014-09-02T11:15:08Z         1      17:1                 site3  
2014-09-10T07:38:14Z         3      18:1                 site4  
2014-09-23T05:28:07Z         2      17:1                 site5  
2014-09-24T15:42:18Z         1      20:1                 site6  
2014-09-08T16:14:48Z         2      17:1                 site7  
2014-09-02T12:09:52Z         2      18:1                 site8  

The password column is only used when you choose to export derived passwords as well, in addition to sites and password types. Anyway, you should be able to find out more about this file format from lhunat's repo: https://github.com/Lyndir/MasterPassword

[djrieger]

Did some research in order to be able to import .mpsites files into the Android app. MPSiteMarshaller.java and mpw-types.h provided some useful information and allowed me to write this Python script. It takes the filename of an .mpsites file as an only command line argument and writes the JSON version of the file contents to standard out:

#!/usr/bin/env python

import sys
import re
from datetime import datetime

mpsitesFile = open(sys.argv[1])
regex = re.compile('^([^# ]+)\s+(\d+)\s+(\d+):(\d+)\s+(\S+)\s*(\S+)?$')
print "["
for line in mpsitesFile:
    if regex.match(line):
        lastUsed, timesUsed, siteType, MPVersion, siteName, sitePassword = regex.match(line).group(1, 2, 3, 4, 5, 6)
        lastUsedJson = datetime.strptime(lastUsed, '%Y-%m-%dT%H:%M:%SZ').strftime("%b %d, %Y %H:%M:%S %p")

        passwordType = {
                        "16": "GeneratedMaximum",
                        "17": "GeneratedLong",
                        "18": "GeneratedMedium",
                        "19": "GeneratedShort",
                        "20": "GeneratedBasic",
                        "21": "PIN",
                        "30": "GeneratedName",
                        "31": "GeneratedPhrase",
                        "1056": "StoredPersonal",
                        "1057": "StoredDevicePrivate"
                       }

        print ("{{\n"
               "    \"category\": \"\",\n"
               "    \"lastUsed\": \"{lastUsed}\",\n"
               "    \"passwordType\": \"{passwordType}\",\n"
               "    \"userName\": \"\",\n"
               "    \"siteName\": \"{siteName}\",\n"
               "    \"siteCounter\": 1\n"
               "}},\n"
               ).format(lastUsed=lastUsedJson, passwordType=passwordType[siteType], siteName=siteName)
print "]"

mpsitesFile.close()

My first attempt at importing a JSON file generated by this script resulted in the Android app crashing. Upon reopening it I saw all passwords had still been imported successfully. Even though, after two seconds the app crashed again.

Some other oddities remain: According to MPSiteMarshaller.java, the third column should have the format siteType:MPVersion:siteCounter, but in .mpsites files exported from the Mac application the siteCounter is omitted, leaving siteType:MPVersion. Moreover, site login names are not exported.

[dkunzler]

Nice work. I saw already that there is now a Java Exporter (Marshaller) for the .mpsites format. I just didn't find the time to include the new Algorithm version into the App. Can you provide a logcat of the crash when importing your generated json file?

[djrieger]

This is my logcat output. It is identical for both the first crash right after/a couple of seconds after the import and subsequent crashes when opening the app again and logging in. It seems though the app only crashes after some scrolling around. Other actions like opening menus or adding new passwords work fine.

I/ActivityManager(  799): Displayed de.devland.masterpassword/.ui.LoginActivity: +365ms (total +886ms)
I/dalvikvm(26427): Could not find method com.melnykov.fab.FloatingActionButton.setElevation, referenced from method com.melnykov.fab.FloatingActionButton.setBackgroundCompat
W/dalvikvm(26427): VFY: unable to resolve virtual method 29860: Lcom/melnykov/fab/FloatingActionButton;.setElevation (F)V
D/dalvikvm(26427): VFY: replacing opcode 0x6e at 0x0013
D/dalvikvm(26427): DexOpt: unable to opt direct call 0x02df at 0x2a in Lcom/melnykov/fab/FloatingActionButton;.setBackgroundCompat
W/dalvikvm(26427): Unable to resolve superclass of Lcom/melnykov/fab/FloatingActionButton$1; (1527)
W/dalvikvm(26427): Link of class 'Lcom/melnykov/fab/FloatingActionButton$1;' failed
D/dalvikvm(26427): DexOpt: unable to opt direct call 0x7465 at 0x2f in Lcom/melnykov/fab/FloatingActionButton;.setBackgroundCompat
I/Timeline(  799): Timeline: Activity_windows_visible id: ActivityRecord{431309e0 u0 de.devland.masterpassword/.ui.LoginActivity t122} time:144737313
D/dalvikvm(26427): Trying to load lib /data/app-lib/de.devland.masterpassword-1/libscrypt.so 0x41faf020
D/dalvikvm(26427): Added shared lib /data/app-lib/de.devland.masterpassword-1/libscrypt.so 0x41faf020
I/Timeline(26427): Timeline: Activity_launch_request id:de.devland.masterpassword time:144755254
I/ActivityManager(  799): START u0 {cmp=de.devland.masterpassword/.ui.PasswordViewActivity} from pid 26427
W/InputMethodManagerService(  799): Window already focused, ignoring focus gain of: com.android.internal.view.IInputMethodClient$Stub$Proxy@42f0ea58 attribute=null, token = android.os.BinderProxy@437e48b8
D/SQL Log (26427): SQLiteQuery: SELECT * FROM SITE ORDER BY SITE_NAME COLLATE NOCASE
D/Sugar   (26427): Fetching properties
D/SQL Log (26427): SQLiteQuery: SELECT * FROM SITE ORDER BY SITE_NAME COLLATE NOCASE
I/dalvikvm(26427): Could not find method it.gmariotti.cardslib.library.view.BaseCardView.drawableHotspotChanged, referenced from method it.gmariotti.cardslib.library.view.CardView.drawableHotspotChanged
W/dalvikvm(26427): VFY: unable to resolve virtual method 31560: Lit/gmariotti/cardslib/library/view/BaseCardView;.drawableHotspotChanged (FF)V
D/dalvikvm(26427): VFY: replacing opcode 0x6f at 0x0000
I/ActivityManager(  799): Displayed de.devland.masterpassword/.ui.PasswordViewActivity: +423ms
I/Timeline(26427): Timeline: Activity_idle id: android.os.BinderProxy@42865b10 time:144755779
I/Timeline(  799): Timeline: Activity_windows_visible id: ActivityRecord{4322d0f0 u0 de.devland.masterpassword/.ui.PasswordViewActivity t122} time:144756018
D/AndroidRuntime(26427): Shutting down VM
W/dalvikvm(26427): threadid=1: thread exiting with uncaught exception (group=0x41cf6db8)
E/AndroidRuntime(26427): FATAL EXCEPTION: main
E/AndroidRuntime(26427): Process: de.devland.masterpassword, PID: 26427
E/AndroidRuntime(26427): java.lang.IllegalArgumentException
E/AndroidRuntime(26427):    at com.google.common.base.Preconditions.checkArgument(Preconditions.java:111)
E/AndroidRuntime(26427):    at com.lyndir.masterpassword.MasterKey.encode(MasterKey.java:89)
E/AndroidRuntime(26427):    at de.devland.masterpassword.util.MasterPasswordHolder.generatePassword(MasterPasswordHolder.java:46)
E/AndroidRuntime(26427):    at de.devland.masterpassword.ui.SiteCard.updatePassword(SiteCard.java:156)
E/AndroidRuntime(26427):    at de.devland.masterpassword.ui.SiteCard.setupInnerViewElements(SiteCard.java:81)
E/AndroidRuntime(26427):    at it.gmariotti.cardslib.library.view.CardView.setupMainView(CardView.java:384)
E/AndroidRuntime(26427):    at it.gmariotti.cardslib.library.view.CardView.buildUI(CardView.java:282)
E/AndroidRuntime(26427):    at it.gmariotti.cardslib.library.view.CardView.setCard(CardView.java:242)
E/AndroidRuntime(26427):    at it.gmariotti.cardslib.library.internal.CardArrayAdapter.getView(CardArrayAdapter.java:171)
E/AndroidRuntime(26427):    at de.devland.masterpassword.util.SiteCardArrayAdapter.getView(SiteCardArrayAdapter.java:139)
E/AndroidRuntime(26427):    at android.widget.AbsListView.obtainView(AbsListView.java:2255)
E/AndroidRuntime(26427):    at android.widget.ListView.makeAndAddView(ListView.java:1790)
E/AndroidRuntime(26427):    at android.widget.ListView.fillDown(ListView.java:691)
E/AndroidRuntime(26427):    at android.widget.ListView.fillGap(ListView.java:655)
E/AndroidRuntime(26427):    at android.widget.AbsListView.trackMotionScroll(AbsListView.java:5143)
E/AndroidRuntime(26427):    at android.widget.AbsListView$FlingRunnable.run(AbsListView.java:4254)
E/AndroidRuntime(26427):    at android.view.Choreographer$CallbackRecord.run(Choreographer.java:761)
E/AndroidRuntime(26427):    at android.view.Choreographer.doCallbacks(Choreographer.java:574)
E/AndroidRuntime(26427):    at android.view.Choreographer.doFrame(Choreographer.java:543)
E/AndroidRuntime(26427):    at android.view.Choreographer$FrameDisplayEventReceiver.run(Choreographer.java:747)
E/AndroidRuntime(26427):    at android.os.Handler.handleCallback(Handler.java:733)
E/AndroidRuntime(26427):    at android.os.Handler.dispatchMessage(Handler.java:95)
E/AndroidRuntime(26427):    at android.os.Looper.loop(Looper.java:136)
E/AndroidRuntime(26427):    at android.app.ActivityThread.main(ActivityThread.java:5146)
E/AndroidRuntime(26427):    at java.lang.reflect.Method.invokeNative(Native Method)
E/AndroidRuntime(26427):    at java.lang.reflect.Method.invoke(Method.java:515)
E/AndroidRuntime(26427):    at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:796)
E/AndroidRuntime(26427):    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:612)
E/AndroidRuntime(26427):    at dalvik.system.NativeStart.main(Native Method)
W/ActivityManager(  799):   Force finishing activity de.devland.masterpassword/.ui.PasswordViewActivity
W/SurfaceFlinger(  224): FB is protected: PERMISSION_DENIED
W/WindowManager(  799): Screenshot failure taking screenshot for (492x874) to layer 21135
D/LogFetchServiceManager( 1707): Received entry added
W/ActivityManager(  799): Activity pause timeout for ActivityRecord{4322d0f0 u0 de.devland.masterpassword/.ui.PasswordViewActivity t122 f}
I/Timeline( 1252): Timeline: Activity_idle id: android.os.BinderProxy@41fa0700 time:144768952
I/Timeline(  799): Timeline: Activity_windows_visible id: ActivityRecord{424425b0 u0 com.cyanogenmod.trebuchet/org.cyanogenmod.trebuchet.CustomHomeLauncher t1} time:144769363
I/Process (26427): Sending signal. PID: 26427 SIG: 9
I/ActivityManager(  799): Process de.devland.masterpassword (pid 26427) has died.
I/WindowState(  799): WIN DEATH: Window{42a0bea8 u0 de.devland.masterpassword/de.devland.masterpassword.ui.PasswordViewActivity}

[djrieger]

Did some more testing. I can scroll around the first 41 sites/passwords, but scrolling one pixel down beyond the 41st site, the app crashes.

[dkunzler]

I can import more than 41 sites without crashing. Since I added MPSITES Export in 1.5.0 I close this issure.