Password no longer recognized

[GoogleCodeExporter]

I'm not sure exactly how to reproduce the problem. I was using Secrets
yesterday when all of a sudden it started rejecting my password. I hadn't
changed my password nor had I updated the Secrets app in a long time.

As far as I can remember, I opened Secrets and entered my password
successfully. I then did a full-text search for a secret. While in the
search results screen I switched away to another app. I might have switched
back to Secrets and then away again multiple times (I don't remember). At
some point later (somewhere between a few minutes to an hour) I tried to go
back to Secrets to look at something else. Since it had been longer than
the timeout, Secrets prompted me for my password. When I entered it it told
me that the password was invalid. Since then it has not accepted my
password despite many tries (and a couple phone reboots too).

I'm using Secrets 1.4.3 on a HTC Dream (aka G1 aka ADP1) running Android
1.5 (CRC1 150275 dream_devphone-userdebug). Let me know what other
information might be useful or if there's something I can run on the
Secrets data file to yield more clues.

Original issue reported on code.google.com by happily....@gmail.com on 2 Sep 2009 at 6:03

[GoogleCodeExporter]

I too am having this issue (4 instances).  The most recent time was when I was 
using
the default music player.  While the music was playing in the background I 
opened up
Secrets.  The G1 paused for a moment and then the music stopped.  I entered 
Secrets
and then exited.  A few minutes later when I tried to open Secrets again, it 
gave me
a wrong password error.  I was using 1.4.3 on a T-mobile G1 which is running CM 
4.0.4
ROM.

I just updated to 1.4.4 and next time I will attempt to get a logcat 
immediately after.

Original comment by rfujim...@gmail.com on 3 Sep 2009 at 3:38

[GoogleCodeExporter]

I pulled out the code needed to read in the secrets file and modified it so 
that it
runs on my laptop and prints out a stack trace (*). Here's the result of 
running it
on the secrets file from my phone:

Exception in thread "main" java.io.EOFException
    at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2297)
    at java.io.ObjectInputStream$BlockDataInputStream.readInt(ObjectInputStream.java:2790)
    at java.io.ObjectInputStream.readHandle(ObjectInputStream.java:1448)
    at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1507)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1749)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1346)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:368)
    at java.util.ArrayList.readObject(ArrayList.java:696)
    at sun.reflect.GeneratedMethodAccessor1.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:616)
    at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:991)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1865)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1770)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1346)
    at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1963)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1887)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1770)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1346)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:368)
    at java.util.ArrayList.readObject(ArrayList.java:696)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:616)
    at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:991)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1865)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1770)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1346)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:368)
    at net.tawacentral.roger.secrets.FileUtils.loadSecrets(FileUtils.java:40)
    at net.tawacentral.roger.secrets.SecretsDumper.main(SecretsDumper.java:22)

I don't know much about Java object serialization. Is it possible to get a copy 
of
the object (in this case ArrayList<Secret>) that contains whatever has been
deserialized right before the exception is thrown?

(*) I'm attaching a zip of the java and class files I used. You'll need to have 
the
BouncyCastle crypto package installed. On Ubuntu 9.04 I had to: 1) install
libbcprov-java and openjdk-6-jdk (sun-java6-jdk doesn't work because it 
requires the
BouncyCastle jar to be signed by a Sun certificate authority and the copy in 
Ubuntu
is not); and 2) add
"security.provider.9=org.bouncycastle.jce.provider.BouncyCastleProvider" to
/etc/java-6-openjdk/security/java.security. Then unzip the file and run "java 
-cp
/usr/share/java/bcprov.jar:. net.tawacentral.roger.secrets.SecretsDumper 
<password>".
It expects a file called "secrets" in the current directory.

Original comment by happily....@gmail.com on 10 Sep 2009 at 11:36

Attachments:

• SecretsDumper.zip

[GoogleCodeExporter]

Thanks for the awesome bug report happily.oblivious!

As you surmise, it seems that there is a problem with java serialization, as if 
the 
file was truncated or was not completely written out on save.  I don't know if 
there 
is a way to get a copy of the object(s) that have been deserialized right 
before the 
exception occurred, but I will investigate.  Sorry for the trouble.

Original comment by roge...@gmail.com on 12 Sep 2009 at 2:30

• Changed state: Accepted

[GoogleCodeExporter]

I create 250 sample secrets to run a test.  With this many secrets, I was able 
to see 
a "force close" message for secrets.  This message happens for any android app 
that 
takes too long to perform a UI action.  In the case of secrets, pausing causes 
it to 
save the secrets, which is taking too long.

I chose "force close" instead of "wait", and the next time I tried logging into 
secrets I got the message that the password was invalid.  Tracing through the 
code, 
the reason is an IO exception, although not an EOF exception as you mention 
above.

Since you do not report seeing the "force close" message, I am going to 
investigate 
whether if there is any way that an application can be force closed without 
asking the 
user first.

Original comment by roge...@gmail.com on 13 Sep 2009 at 3:31

[GoogleCodeExporter]

I was in the middle of doing a few things at once. So while I don't think I got 
a
"force close" message, it's possible that there was one that I dismissed and 
then
forgot about.

Also, the EOF exception above is from running the java code on my laptop, not 
on the
phone. Maybe Android's implementation of java serialization throws an IO 
exception at
a different point in the de-serialization process than Sun's java?

Lastly, I decrypted the corrupt secrets file to see what state it was in. It
definitely looks like a truncation after the 92nd record in the file (I 
probably had
around 200 records total). I guess I'll have to hope that most of the other 100
records haven't been added or changed much since my last backup a long time ago.

Original comment by happily....@gmail.com on 14 Sep 2009 at 6:10

[GoogleCodeExporter]

Hi happily.oblivious,

All that you say is correct.

I have fixed secrets to be much more robust to such failures, you can take a 
look at 
revisions 57 to 60.  Version 1.4.6 in the download section of this site has 
these fixes, 
and I will shortly upload it to the market after a little more testing.  
Basically, it 
now writes to a temp file first, and overwrites the original secrets file only 
once 
everything succeeds.  Also, the writing is done in a background thread to 
prevent the 
"force close" messages.

This does not help you or people whose data has already been corrupted though. 
I am now 
working to see if/how to recover as much as possible from a corrupted secrets 
file.  Are 
you saying that you already wrote code that does this?  That is, have you 
recovered your 
first 92 records?  If so, would you mind sharing that code so that others could 
benefit?  
Thanks!

Original comment by roge...@gmail.com on 17 Sep 2009 at 3:20

[GoogleCodeExporter]

All I did is tweak the loadSecrets() function in the code I attached earlier to 
dump
the CipherInputStream containing the secrets into a file after decrypting it 
instead
of passing it to ObjectInputStream. So I got a decrypted copy of the serialized 
java
object. Running the unix "strings" command on that was enough to see the 
contents of
the file and figure out what was in it. I didn't figure out how to get
ObjectInputStream to decode the non-corrupt subset of the stream, but the 
strings
output is good enough for me as far as recovery goes.

Thanks for the recent changes! I skimmed over them and they look like the right
approach. What do you think about changing the file format to something more 
portable
(i.e. xml/yaml/json/etc. rather than java serialization)? It would give people 
more
confidence that they could get their data out in these kinds of situations. 
Plus it'd
allow writing programs in other languages to process the secrets outside of the 
phone
(e.g. a desktop app), if someone wanted to do so.

Original comment by happily....@gmail.com on 17 Sep 2009 at 4:00

[GoogleCodeExporter]

I have made the fix to allow for recovery of secrets.  On your phone please go 
to the 
secrets website at http://code.google.com/p/secrets-for-android/ and download 
the new 
version 1.4.7.  Then start it and login using your correct password.  If all is 
good, 
you should see a message telling there was a problem during load and some data 
may be 
lost.  After that you should see all recovered secrets.  Note that any missing 
secrets 
are truly lost and you will need to enter them again.

Let me know how it goes.  Again, sorry for the trouble.

Original comment by roge...@gmail.com on 21 Sep 2009 at 1:43

[GoogleCodeExporter]

Fixed in revision 62.

Original comment by roge...@gmail.com on 21 Sep 2009 at 11:23

• Changed state: Fixed

[GoogleCodeExporter]

Thanks Roger. The new recovery code works (and good idea for how to do it!) and 
was
able to recover the 92 records that were in my old file.

Original comment by happily....@gmail.com on 23 Sep 2009 at 12:04

[GoogleCodeExporter]

Thanks for testing happily.oblivious, and glad it worked out for the 92 
records.  
Sorry about the other records.  I will put this up on the market shortly.

Original comment by roge...@gmail.com on 23 Sep 2009 at 1:15