ccavanaugh / jgnash

jGnash Personal Finance
http://ccavanaugh.github.io/jgnash/
Other
139 stars 80 forks source link

Quicken categories.qif uncaught exception #30

Open matteg opened 7 years ago

matteg commented 7 years ago

I am trying to transfer historical data from Quicken 2015 into jGnash version 2.32.0 My Quicken accounts transferred with no difficulty I imported transactions for a quicken account without difficulty although they were treated as one sided transactions (which means I need to eventually edit each one manually)

Given that Quicken categories are treated in jGnash as Accounts, I was hoping that importing a categories.qif file would create Expense or Income Accounts in jGnash. However when I attempted an import on Ubuntu Linux there was no prompt for date format and I received an uncaught exception: java.lang.NullPointerException at java.util.regex.Matcher.getTextLength(Matcher.java:1283) at java.util.regex.Matcher.reset(Matcher.java:309) at java.util.regex.Matcher.(Matcher.java:229) at java.util.regex.Pattern.matcher(Pattern.java:1093) at java.util.regex.Pattern.split(Pattern.java:1206) at java.util.regex.Pattern.split(Pattern.java:1273) at jgnash.convert.imports.qif.QifTransaction.determineDateFormat(QifTransaction.java:87) at jgnash.convert.imports.qif.QifAccount.getTransactions(QifAccount.java:43) at jgnash.convert.imports.qif.QifParser.parsePartialFile(QifParser.java:171) at jgnash.convert.imports.qif.QifParser.parsePartialFile(QifParser.java:98) at jgnash.convert.imports.qif.QifImport.doPartialParse(QifImport.java:108) at jgnash.ui.actions.ImportQifAction.importQif(ImportQifAction.java:137) at jgnash.ui.actions.ImportQifAction.actionPerformed(ImportQifAction.java:195) at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:2022) at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2348) at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:402) at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:259) at javax.swing.AbstractButton.doClick(AbstractButton.java:376) at javax.swing.plaf.basic.BasicMenuItemUI.doClick(BasicMenuItemUI.java:833) at javax.swing.plaf.basic.BasicMenuItemUI$Handler.mouseReleased(BasicMenuItemUI.java:877) at java.awt.Component.processMouseEvent(Component.java:6533) at javax.swing.JComponent.processMouseEvent(JComponent.java:3324) at java.awt.Component.processEvent(Component.java:6298) at java.awt.Container.processEvent(Container.java:2236) at java.awt.Component.dispatchEventImpl(Component.java:4889) at java.awt.Container.dispatchEventImpl(Container.java:2294) at java.awt.Component.dispatchEvent(Component.java:4711) at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4888) at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4525) at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4466) at java.awt.Container.dispatchEventImpl(Container.java:2280) at java.awt.Window.dispatchEventImpl(Window.java:2746) at java.awt.Component.dispatchEvent(Component.java:4711) at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:758) at java.awt.EventQueue.access$500(EventQueue.java:97) at java.awt.EventQueue$3.run(EventQueue.java:709) at java.awt.EventQueue$3.run(EventQueue.java:703) at java.security.AccessController.doPrivileged(Native Method) at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:80) at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:90) at java.awt.EventQueue$4.run(EventQueue.java:731) at java.awt.EventQueue$4.run(EventQueue.java:729) at java.security.AccessController.doPrivileged(Native Method) at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:80) at java.awt.EventQueue.dispatchEvent(EventQueue.java:728) at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:201) at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116) at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105) at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101) at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93) at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)

matteg commented 7 years ago

I repeated the test using jgnash2.exe on windows 10 with similar results: java.lang.NullPointerException at java.util.regex.Matcher.getTextLength(Unknown Source) at java.util.regex.Matcher.reset(Unknown Source) at java.util.regex.Matcher.(Unknown Source) at java.util.regex.Pattern.matcher(Unknown Source) at java.util.regex.Pattern.split(Unknown Source) at java.util.regex.Pattern.split(Unknown Source) at jgnash.convert.imports.qif.QifTransaction.determineDateFormat(QifTransaction.java:87) at jgnash.convert.imports.qif.QifAccount.getTransactions(QifAccount.java:43) at jgnash.convert.imports.qif.QifParser.parsePartialFile(QifParser.java:171) at jgnash.convert.imports.qif.QifParser.parsePartialFile(QifParser.java:98) at jgnash.convert.imports.qif.QifImport.doPartialParse(QifImport.java:108) at jgnash.ui.actions.ImportQifAction.importQif(ImportQifAction.java:137) at jgnash.ui.actions.ImportQifAction.actionPerformed(ImportQifAction.java:195) at javax.swing.AbstractButton.fireActionPerformed(Unknown Source) at javax.swing.AbstractButton$Handler.actionPerformed(Unknown Source) at javax.swing.DefaultButtonModel.fireActionPerformed(Unknown Source) at javax.swing.DefaultButtonModel.setPressed(Unknown Source) at javax.swing.AbstractButton.doClick(Unknown Source) at javax.swing.plaf.basic.BasicMenuItemUI.doClick(Unknown Source) at javax.swing.plaf.basic.BasicMenuItemUI$Handler.mouseReleased(Unknown Source) at java.awt.Component.processMouseEvent(Unknown Source) at javax.swing.JComponent.processMouseEvent(Unknown Source) at java.awt.Component.processEvent(Unknown Source) at java.awt.Container.processEvent(Unknown Source) at java.awt.Component.dispatchEventImpl(Unknown Source) at java.awt.Container.dispatchEventImpl(Unknown Source) at java.awt.Component.dispatchEvent(Unknown Source) at java.awt.LightweightDispatcher.retargetMouseEvent(Unknown Source) at java.awt.LightweightDispatcher.processMouseEvent(Unknown Source) at java.awt.LightweightDispatcher.dispatchEvent(Unknown Source) at java.awt.Container.dispatchEventImpl(Unknown Source) at java.awt.Window.dispatchEventImpl(Unknown Source) at java.awt.Component.dispatchEvent(Unknown Source) at java.awt.EventQueue.dispatchEventImpl(Unknown Source) at java.awt.EventQueue.access$500(Unknown Source) at java.awt.EventQueue$3.run(Unknown Source) at java.awt.EventQueue$3.run(Unknown Source) at java.security.AccessController.doPrivileged(Native Method) at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source) at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source) at java.awt.EventQueue$4.run(Unknown Source) at java.awt.EventQueue$4.run(Unknown Source) at java.security.AccessController.doPrivileged(Native Method) at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source) at java.awt.EventQueue.dispatchEvent(Unknown Source) at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source) at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source) at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source) at java.awt.EventDispatchThread.pumpEvents(Unknown Source) at java.awt.EventDispatchThread.pumpEvents(Unknown Source) at java.awt.EventDispatchThread.run(Unknown Source)

ccavanaugh commented 7 years ago

Can you provide a snippet of a file with the date? QIF is broken and the date format is not standard. Can you export OFX/QFX files from Quicken? Results will be much better.

matteg commented 7 years ago

I’m happy to provide the entire categories.qif file but I’d rather not have it publically available.

Is there a way I can send it directly to you (only) ?

The date in my Quicken 2015 QIF files is as follows:

There are no dates in my AccountList.qif or my AccountCategoryList.QIF file …

In my AccountTransactions.QIF a snippet is

!Clear:AutoSwitch

!Option:AutoSwitch

!Account

NChequing-Gerry-HSBC

DGerry's Personal Account

TBank

^

!Type:Bank

D1/ 1' 5

U10,229.75

T10,229.75

CX

POpening Balance

L[Chequing-Gerry-HSBC]

^

D1/ 4' 5

U-49.75

T-49.75

CX

P*POS Purchase

MFAMOUS PLAYERS #0686 #0001002046

LEntertainment

^

And also (to show a day number January 16, 2006:

D1/16' 5

U-4.00

T-4.00

CX

P*SERVICE Charge

MBASIC BANKING PACKAGE 16DEC04 TO 16JAN05

LFinance Charges:Banking Package

^

Gerry

Gerry

From: Craig Cavanaugh [mailto:notifications@github.com] Sent: August 23, 2017 3:17 PM To: ccavanaugh/jgnash Cc: Gerry Matte; Author Subject: Re: [ccavanaugh/jgnash] Quicken categories.qif uncaught exception (#30)

Can you provide a snippet of a file with the date? QIF is broken and the date format is not standard. Can you export OFX/QFX files from Quicken? Results will be much better.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ccavanaugh/jgnash/issues/30#issuecomment-324478348 , or mute the thread https://github.com/notifications/unsubscribe-auth/AChK95sXeyIgcrn_uA9yIQuBuqL15hp3ks5sbKTOgaJpZM4O_LRY . https://github.com/notifications/beacon/AChK93n3SJ-sAuReRLzMdPLvwOwgEujiks5sbKTOgaJpZM4O_LRY.gif

matteg commented 7 years ago

Sorry … Quicken 2015 will only export or import QFX files of it’s entire database and it is a binary (or encrypted ?) format.

Gerry

From: Craig Cavanaugh [mailto:notifications@github.com] Sent: August 23, 2017 3:17 PM To: ccavanaugh/jgnash Cc: Gerry Matte; Author Subject: Re: [ccavanaugh/jgnash] Quicken categories.qif uncaught exception (#30)

Can you provide a snippet of a file with the date? QIF is broken and the date format is not standard. Can you export OFX/QFX files from Quicken? Results will be much better.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ccavanaugh/jgnash/issues/30#issuecomment-324478348 , or mute the thread https://github.com/notifications/unsubscribe-auth/AChK95sXeyIgcrn_uA9yIQuBuqL15hp3ks5sbKTOgaJpZM4O_LRY . https://github.com/notifications/beacon/AChK93n3SJ-sAuReRLzMdPLvwOwgEujiks5sbKTOgaJpZM4O_LRY.gif

ccavanaugh commented 7 years ago

OFX is an open standard text based standard and QFX should only contain a couple of extra fields.

I understand completely about not sharing a complete file. Let's see what I can do with the snippet first.

And also (to show a day number January 16, 2006: D1/16' 5

Is this a typo and the year is actually 2005?

matteg commented 7 years ago

I’m not sure what you mean by saying QIF is broken …… My files comply with https://en.wikipedia.org/wiki/Quicken_Interchange_Format

The dates in my files follow the first entry in the section “Detail Items”

I was able to write my own QIF import program (to H2 and to mySql) but the jGnash table structures are complex enough that I would rather use jGnash imports if they can be made to work …..

Thanks

Gerry

From: Craig Cavanaugh [mailto:notifications@github.com] Sent: August 23, 2017 3:17 PM To: ccavanaugh/jgnash Cc: Gerry Matte; Author Subject: Re: [ccavanaugh/jgnash] Quicken categories.qif uncaught exception (#30)

Can you provide a snippet of a file with the date? QIF is broken and the date format is not standard. Can you export OFX/QFX files from Quicken? Results will be much better.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ccavanaugh/jgnash/issues/30#issuecomment-324478348 , or mute the thread https://github.com/notifications/unsubscribe-auth/AChK95sXeyIgcrn_uA9yIQuBuqL15hp3ks5sbKTOgaJpZM4O_LRY . https://github.com/notifications/beacon/AChK93n3SJ-sAuReRLzMdPLvwOwgEujiks5sbKTOgaJpZM4O_LRY.gif

matteg commented 7 years ago

Yes – I caught the typo too .. the year was supposed to represent 2005 not 2006 (my bad)

I did an export to QFX from Quicken 2015 and then opened the output with a text editor:

From: Craig Cavanaugh [mailto:notifications@github.com] Sent: August 23, 2017 4:09 PM To: ccavanaugh/jgnash Cc: Gerry Matte; Author Subject: Re: [ccavanaugh/jgnash] Quicken categories.qif uncaught exception (#30)

OFX is an open standard text based standard and QFX should only contain a couple of extra fields.

I understand completely about not sharing a complete file. Let's see what I can do with the snippet first.

And also (to show a day number January 16, 2006: D1/16' 5

Is this a typo and the year is actually 2005?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ccavanaugh/jgnash/issues/30#issuecomment-324487633 , or mute the thread https://github.com/notifications/unsubscribe-auth/AChK96r0PZ32dYZFBU0AWX8eVZ3wMrzGks5sbLEBgaJpZM4O_LRY . https://github.com/notifications/beacon/AChK9_wuC1f6M3eolRTderYb3DXl0DL6ks5sbLEBgaJpZM4O_LRY.gif

ccavanaugh commented 7 years ago

No idea why an OFX would be binary... that would be a first for me. Anyway...

The existing code should handle the date format correctly, but it appears as there is an offending transaction without a date triggering the null pointer exception and preventing the import from getting far enough to display the dialog.

Are you able to compile and run from code? If not, you can send the test file to jgnash.devel@gmail.com

matteg commented 7 years ago

Quicken only exports to a QXF file in binary format. All data is exported. That file can then be imported into a fresh Quicken installation – a process I used when I implemented a new PC a few years ago.

When I download transactions from my bank in Quicken format, the result is a textual QXF file with headers that seem to comply with the OFX format. It imports into Quicken 2015 with no problems.

OFXHEADER:100

DATA:OFXSGML

VERSION:102

SECURITY:NONE

ENCODING:USASCII

CHARSET:1252

COMPRESSION:NONE

OLDFILEUID:NONE

NEWFILEUID:NONE

0 INFO OK 20170823195451.000[-4:EDT] ENG 00025 [snip] Yes, a couple of hours ago I downloaded the current source, compiled it, and then executed jGnash2.exe. It loaded my data file with no apparent errors. I will send the failing Quicken QIF file AccountCategoryList.QIF to the email address you supplied. Gerry From: Craig Cavanaugh [mailto:notifications@github.com] Sent: August 23, 2017 4:52 PM To: ccavanaugh/jgnash Cc: Gerry Matte; Author Subject: Re: [ccavanaugh/jgnash] Quicken categories.qif uncaught exception (#30) No idea why an OFX would be binary... that would be a first for me. Anyway... The existing code should handle the date format correctly, but it appears as there is an offending transaction without a date triggering the null pointer exception and preventing the import from getting far enough to display the dialog. Are you able to compile and run from code? If not, you can send the test file to jgnash.devel@gmail.com — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub , or mute the thread .