Doesn't convert Keychain Notes

[MacLemon]

Summary:

Notes from Keychain.app can't be converted properly.

Steps to Reproduce:

I'm using a .keychain containing 115 text notes and nothing else to test migration from Keychain to 1Password. I've exported said .keychain as suggested in the README with security dump-keychain -d test.keychain > test.txt This resulted in a text file with all the items sorted by modification date and looks like the expected output form the security command.

Running convert_to_1p4 like this:

./convert_to_1p4.pl keychain -v test.txt gives me this output:

Examined 115 records
Skipped 113 non-login records
Skipped 0 duplicate records
Imported 2 records
Exported 2 login items
Exported 2 total items

113 of 115 notes items are completely skipped. The remaining 2 items are exported into the `.1pif`` file as password items.

Changing the export to convert_to_1p4.pl keychain -i note -e note -v test.txt results in a completely empty .1pif file.

Debug output for skipped items looks like this:

main                : Command Line: keychain -d test.txt
main                : Output file: /Users/<usershortname>/Desktop/1P4_import.1pif
do_import           : Entry 1
do_import           :   considering rules for CLASS
do_import           :       rule 1: called with genp
do_import           :       rule 1: returns 0   genp
do_import           :   considering rules for svce
do_import           :       rule 1: called with "Item - Title"
do_import           :       rule 1: returns 0   "Item - Title"
do_import           :       rule 2: called with "Item - Title"
do_import           :       rule 2: returns 1   Item - Title
do_import           :       rule 3: called with Item - Title
do_import           :       rule 3: returns 0   Item - Title
do_import           :   considering rules for srvr
do_import           :   considering rules for path
do_import           :   considering rules for ptcl
do_import           :   considering rules for acct
do_import           :       rule 1: called with <NULL>
do_import           :       rule 1: returns 0   <NULL>
do_import           :       rule 2: called with <NULL>
do_import           :       rule 2: returns 0   <NULL>
do_import           :   considering rules for mdat
do_import           :       rule 1: called with 0x32303034303531363135333134355A00  "20040516153145Z\000"
do_import           :       rule 1: returns 1   2004-05-16 15:31:45
do_import           :   considering rules for cdat
do_import           :       rule 1: called with 0x32303034303531363135333134315A00  "20040516153141Z\000"
do_import           :       rule 1: returns 1   2004-05-16 15:31:41
do_import           :   considering rules for DATA
do_import           :       rule 1: called with 0x<bunch of hex digits>...
do_import           :       rule 1: returns 1   0x<bunch of hex digits>...
__ANON__            :           skipping non-password record: genp: Item - Title

Expected Results:

Notes should be exported as notes to be imported as notes again. Notes should not be skipped.

Actual Results:

Notes aren't converted to notes but logins. Most notes are completly skipped.

Regression:

Tried Perl 5.16 included in OS X Mavericks as well as perl5.16 built with MacPorts. Explicitly specifying the perl interpreter doesn't change any of the results. Keychain.app doesn't find any problems with the .keychain file.

Specifying -i note results in the same 2 passwords ending up in the .1pif. Specifying -e note results in an empty export file. (0 bytes) Specyfing both results in an empty export file.

Notes:

OS X 10.9.5 (13F34) Keychain.app Version 9.0 (55153) 1Password 4.4.3 (443000) convert_to_1p4 1.0.2 (From Agilebits Website, as well as from GitHub.)

[mikecappella]

MacLemon,

Thanks for your report. I've updated the Keychain converter to support Secure Notes. The version is posted in my _testing folder. See the thread:

https://discussions.agilebits.com/discussion/30286/mrcs-convert-to-1password-utility/p1

For instructions. Please test out the new version, and add your comments here or within that thread. Best, MrC

[MacLemon]

Test results with 1.0.3:

Examined 115 records
Skipped 30 non-login records
Skipped 0 duplicate records
Imported 85 records
Exported 85 note items
Exported 85 total items

Debug output:

main                : Command Line: keychain -d /Users/<usershortname>/Desktop/test.txt
main                : Output file: /Users/<usershortname>/Desktop/1P4_import.1pif
do_import           : Entry 1
do_import           :   considering rules for CLASS
do_import           :       rule 1: called with genp
do_import           :       rule 1: returns 0   genp
do_import           :   considering rules for svce
do_import           :       rule 1: called with "Item - Title"
do_import           :       rule 1: returns 0   "Item - Title"
do_import           :       rule 2: called with "Item - Title"
do_import           :       rule 2: returns 1   Item - Title
do_import           :       rule 3: called with Item - Title
do_import           :       rule 3: returns 0   Item - Title
do_import           :   considering rules for srvr
do_import           :   considering rules for path
do_import           :   considering rules for ptcl
do_import           :   considering rules for acct
do_import           :       rule 1: called with <NULL>
do_import           :       rule 1: returns 0   <NULL>
do_import           :       rule 2: called with <NULL>
do_import           :       rule 2: returns 0   <NULL>
do_import           :   considering rules for mdat
do_import           :       rule 1: called with 0x32303034303531363135333134355A00  "20040516153145Z\000"
do_import           :       rule 1: returns 1   2004-05-16 15:31:45
do_import           :   considering rules for cdat
do_import           :       rule 1: called with 0x32303034303531363135333134315A00  "20040516153141Z\000"
do_import           :       rule 1: returns 1   2004-05-16 15:31:41
do_import           :   considering rules for desc
do_import           :       rule 1: called with "secure note"
do_import           :       rule 1: returns secure note   secure note
do_import           :   considering rules for DATA
do_import           :       rule 1: called with 0x<bunch of hex digits>...
do_import           :       rule 1: returns 0   0x<bunch of hex digits>...
do_import           :       rule 2: called with 0x<bunch of hex digits>...
do_import           :       rule 2: returns 1   0x<bunch of hex digits>...
__ANON__            :           skipping non-password record: genp: Item - Title

Some items still get skipped.

Best regards MacLemon

[mikecappella]

Can you help me identify the pattern of the note in the "DATA" section? I'm expecting data for notes that follows the pattern:

0x<A SEQUENCE OF HEXDIGITS IN CAPS><WHITESPACE>"<ANYTHING><key>NOTE</key>\012\011<string><CAPTURED NOTE></string><ANYTHING>

It would seem some of your notes do not match this pattern, but I'm not sure how to generate this.

If you understand regular expressions, it might be obvious to you in the code what is missing or needs to be changed. Or maybe you could help me understand the pattern without revealing any confidential data.

Feel free to reply at my email address at the top of the script.

[MacLemon]

Would greatly help to make debug optionally output the items it skips and why. Would certainly help me to identify patterns of the notes skipped. I guess I could derive generic items for you to test with then. My Perl foo is quite weak.

[mikecappella]

@MacLemon,

I don't monitor this Issues area much - did the update resolve your issues?

[MacLemon]

Nope, didn't resolve the issue.

[mikecappella]

Pepi,

I’ve lost track of, or can’t recall what the issue was with the Keychain converter you mentioned here:

https://github.com/AgileBits/onepassword-utilities/issues/4 <https://github.com/AgileBits/onepassword-utilities/issues/4>

Was this just a matter that some Notes weren’t converted, and you want extra debug info? Perhaps you can elaborate and I’ll fix it this weekend.

Best, MrC

On May 4, 2015, at 10:53 AM, Pepi Zawodsky notifications@github.com wrote:

Nope, didn't resolve the issue.

— Reply to this email directly or view it on GitHub https://github.com/AgileBits/onepassword-utilities/issues/4#issuecomment-98796130.

[eppfel]

I don't know if this related to the problem of @MacLemon , but in my case Notes did not import, because of language. I use a german Version of OSX and therefore the desc attribute reads "Sichere Notiz".

Changing this in Keychain.pem#L91 worked for me.

I was wondering: Why does the type attribute not satisfy as condition?

Thx, for the great tool, btw.

[mikecappella]

Hi @dharma-guardian ,

Thank you very much for the info. I was not aware of the localized attribute values. I've updated the 1.09 version in Testing Bits - see the thread:

https://discussions.agilebits.com/discussion/30286/mrcs-convert-to-1password-utility/p1

and give the new bits a try!

[eppfel]

LGTM: All 19 secure notes were imported without problems.

[cacheguard]

Hi there I had the same issue with a Mac in French and the v 1.09 saved my life :-) Thank you so much for your great job. Best Regards

[eppfel]

I think, if @MacLemon does not reply, this seems to be the fix and the issues can be closed.