search

t1794t7586 / mdcsvimporter

Automatically exported from code.google.com/p/mdcsvimporter
GNU Lesser General Public License v3.0
0 stars 0 forks source link

Unsupported CSV file format - documentation needed? #22

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
I am trying to import data from DBS bank, copying from the table of 
transactions and pasting them in a CSV file. No matter how many variations I 
have tried, I cannot get the dialog to get past the message "Unsupported CSV 
file format." Here is an example of a line in the file, with a header added: 

"Date","Transaction type","Description","Debit","Credit"
"2012-11-29","POS","Fair Price","S$25.12",""

I am using Moneydance v. 2012 (825) on Mac OSX Lion.

Original issue reported on code.google.com by dcowen...@gmail.com on 15 Dec 2012 at 5:51

GoogleCodeExporter commented 9 years ago 
Hi dcowen,
I was able to import your posted example.  First I had to remove the 's' in 
front of
 the $25.12 debit.
I set up the custom reader per the attached screenshot.
HTH -Kevin N.

Original comment by kmnugent...@gmail.com on 16 Dec 2012 at 6:42

Attachments:

GoogleCodeExporter commented 9 years ago 
Hi dcowan,
You may need to swap the -Payment- and -Deposit- settings.
-Kevin N.

Original comment by kmnugent...@gmail.com on 16 Dec 2012 at 6:51

GoogleCodeExporter commented 9 years ago 
Thank you, Kevin, for showing me the settings. I did just what you suggested, 
but I still get the same error message. I am attaching the file I am using to 
see if that is the source of the error.

Original comment by dcowen...@gmail.com on 17 Dec 2012 at 1:14

Attachments:

GoogleCodeExporter commented 9 years ago 
Hi dcowan,

I opened your posted 'test.csv' in Excel.  When saving a file as a .csv in 
Excel it
warns of some format changes before allowing the user to save it.  I ignored 
those
warnings and went ahead and saved it.

I did notice though that opening it in Excel, the date format was MM/DD/YYYY.  
If I 
opened it in a simple text editor (Notepad) the date format was YYYY-DD-MM.

Try setting up the custom reader per the new attached screenshot.  Take note of 
the
date format changes MM/DD/YYYY (the order is changed and the dashes are now 
forward
slashes) 

I was able to process test.csv with these settings.  (See Attached)

HTH -Kevin N.

Original comment by kmnugent...@gmail.com on 17 Dec 2012 at 2:27

Attachments:

GoogleCodeExporter commented 9 years ago 
I never had a problem with the importer until I started using Windows 8.  Now 
all I get is "unsupported csv file format".  It works fine with all previous 
Windows OS.

Original comment by marthald...@gmail.com on 13 Jan 2013 at 6:30

GoogleCodeExporter commented 9 years ago 
Hi. I did not get any email about these defects so I did not know about them.
Anyways, I just tried this one and it works for me using MD 2012.3
I used your text from above and your mac csv zip file. Both ways works.
I created a word doc attached that shows what I did. See if that helps.
Let me know if this is still a problem.
P.S. I did recently put out a download 'How To...' doc you can get as well.

Original comment by stashu....@gmail.com on 23 Jan 2013 at 4:51

Attachments:

GoogleCodeExporter commented 9 years ago 
Thank you for looking into this. It still will not recognize the file type. See 
the attached screenshots of what I have done.

Original comment by dcowen...@gmail.com on 13 Feb 2013 at 1:08

Attachments:

GoogleCodeExporter commented 9 years ago 
Hi dcowen,  Have you tried it with the 'check number' field set to 'Can be 
blank'?
-Kevin N.

Original comment by kmnugent...@gmail.com on 13 Feb 2013 at 3:42

GoogleCodeExporter commented 9 years ago 
Hi.  Look at your date format: It is wrong I think.
You have 'YYY instead of YYYY

Stan

Original comment by stashu....@gmail.com on 13 Feb 2013 at 4:01

GoogleCodeExporter commented 9 years ago 
Hmmm. Your font looks just bigger. Maybe you do have YYYY, but
then we cannot see your File Encoding: Make your window bigger
and see if you have UTF-8 like I have.
Otherwise look in your 'console' window. It will tell you exactly
why it stopped. Look for when it is testing your DBS reader and
see what it did not like. You can send me your console output if
you want. Hide your real data. Give an example that does not work.
And 3rd, as Kevin said, if a field can be blank, you have to say that.

Stan

Original comment by stashu....@gmail.com on 13 Feb 2013 at 4:06

GoogleCodeExporter commented 9 years ago 
The date format is correct. The font is just too large, and resizing the window 
does not solve the problem. But that is another issue.

This is a basic question, but where do I find the console window? Do you need 
me to start the program in the terminal and watch the output?

I tried making sure that the check number field could be blank (which it is not 
in the example data), but not joy.

One potential clue is that whenever I close the import file dialog and open it 
again, my custom file readers are deleted. In other words, I have to create a 
new one every time.

Original comment by dcowen...@gmail.com on 13 Feb 2013 at 5:58

GoogleCodeExporter commented 9 years ago 
Go to Help -> Console Window

it will open and tell you some things:
Starting Moneydance 2012.3 (852) at Wed Feb 13 08:03:12 EST 2013
Settings Folder: C:\Users\stan\.moneydance                <-- this will be 
different for Mac. Look in here
OS: Windows 7 6.1 amd64
Java: 1.7.0_07 (Oracle Corporation)
Is in sandbox: false; restricted file access: false
name and version =CSV Importer Beta 15.7.8=
opening last file: c:\.....\my-ledger.moneydance
Dropbox interface: DBAPI:/.moneydancesync/....

What does yours say?

As for your readers not saving, something is wrong.
Do you define a custom reader and hit 'Add'?

Your settings for all custom readers are in this file:
C:\Users\stan\.moneydance\mdcsvimporter.props   or what yours is on your Mac.
Do you have rights to that folder and file?

You should see lines in this file like this. You can actually copy these lines 
into
your file, save it, and you should get a custom reader called: issue22

#MDCSVImporter - Moneydance CSV Importer
#Mon Feb 11 15:17:54 EST 2013
reader\:issue22.FileEncodingString=UTF-8
reader\:issue22.DateFormatString=YYYY-MM-DD
reader\:issue22.FieldSeparator=44
reader\:issue22.FooterLines=0
reader\:issue22.AmountDecimalSignChar=46
reader\:issue22.ImportReverseOrderFlag=false
reader\:issue22.DataTypesList=[date, check number, description, -Deposit-, 
-Payment-, , , , , ]
reader\:issue22.EmptyFlagsList=[, , , Can Be Blank, Can Be Blank, , , , , ]
reader\:issue22.AmountGroupingSeparatorChar=44
reader\:issue22.HeaderLines=1
reader\:issue22.Name=issue22
reader\:issue22.AmountCurrencyChar=36
reader\:issue22.AmountFormat=\#,\#\#\#,\#\#\#,\#\#0.00;(\#)

because you are on a Mac your end of line character is different and copying
these lines in may or may not work. If your editor leaves these as separate 
lines
it will probably work. It is just a simple text file though, that you can modify
yourself.

Stan

Original comment by stashu....@gmail.com on 13 Feb 2013 at 1:20

GoogleCodeExporter commented 9 years ago 
You mention a few issues. I will answer one at a time.

1. Yes, when creating the importer I clicked on Add and then Save. It then 
appears in the List of Readers.

2. I looked for a file named ".moneydance" (with hidden files showing) and did 
not find one. 

3. My console output is attached. Moneydance seems to be looking for a folder 
/Users/[username]/.moneydance/ but not finding it.

Original comment by dcowen...@gmail.com on 14 Feb 2013 at 12:55

Attachments:

GoogleCodeExporter commented 9 years ago 
Well,
java.io.FileNotFoundException: 
/Users/[username]/.moneydance/mdcsvimporter.props (No such file or directory)  
is a problem.

I'm not so much of a Mac person, but on my wife's mac, it works, and instead of 
[username] is has her login name. Why does that not work for you? You will have 
to figure that out. My reader wants to save to this file: 
/Users/[username]/.moneydance/mdcsvimporter.props
This is the [User.Home] folder then /.moneydance/mdcsvimporter.props

1.) I think you have to create a mac username for yourself. Fixing this might 
fix it all because I do try to create the mdcsvimporter.props file.
Or if you can set your user.home folder that might work.
You might even try creating this folder /Users/[username]  (not the best fix)
but I do not know if it will allow these characters [ or ].

2.) after that just make sure when you start MD this 
[User.Home]/.moneydance/mdcsvimporter.props   can be created. You can create an 
empty file if you want, but the custom reader will probably create it for you. 
Let me know.

Original comment by stashu....@gmail.com on 14 Feb 2013 at 3:49

GoogleCodeExporter commented 9 years ago 
I will try to create the file manually, but keep in mind that I replaced my 
real username with [username]. So Moneydance is looking in the right place, but 
the program did not actually create that file in the first place. Is this a bug 
in Moneydance? It seems to me the user settings folder should be created 
automatically...

Original comment by dcowen...@gmail.com on 14 Feb 2013 at 3:58

GoogleCodeExporter commented 9 years ago 
So I created the folder and file manually. When I click "Add" there is a pause 
(there wasn't before), so I thought that it was doing something. But when I 
look at the file, nothing is changed. There is still the line:

java.io.FileNotFoundException: /Users/daniel/.moneydance/mdcsvimporter.props

But I tried:

bash$ echo data >> /Users/daniel/.moneydance/mdcsvimporter.props

The word "data" is appended to the file. So I have created the right file in 
the right place, but Moneydance is not finding it. I am attaching a new file 
with console output.

Original comment by dcowen...@gmail.com on 14 Feb 2013 at 4:54

Attachments:

GoogleCodeExporter commented 9 years ago 
Hi. I wondered what happened. The system did not send me your message.
1.) So, as I mentioned, my code I am pretty sure does try to create the 
settings file.
2.) I am a little confused. You said you used the name [username] but I see 
"daniel". using [ or ] might not work as a file/name.
3.) Until you get a custom reader 'save' to work you will have a problem.
4.) Now it looks like a rights problem on the settings file:
java.io.FileNotFoundException: /Users/daniel/.moneydance/mdcsvimporter.props 
(Operation not permitted)

whatever user you are logging in with must be able to read and write to the 
settings file.
if you cd /Users/daniel/.moneydance
ls -l mdcsvimporter.props
you want to see rights on that file something like this:
drwxrwx--- daniel daniel(a group) mdcsvimporter.props

change rights on the settings file so your login user can read and write it.
and if daniel is your user

Original comment by stashu....@gmail.com on 15 Feb 2013 at 2:53

GoogleCodeExporter commented 9 years ago 
1.) That is odd. I wonder if there is a mac issue here.
2.) I decided in the second case not to clean up the personally identifiable 
information (which is minimal). I had sanitized the output in the first case, 
so don't be confused. The second post of Console.txt is unedited. 
3.) I figured as much.
4.) I created the file in the same user. Changing permissions to rw for all 
does nothing.

Original comment by dcowen...@gmail.com on 18 Feb 2013 at 12:22

GoogleCodeExporter commented 9 years ago 
Hi. I took a look on a Mac pc and updated them from MD 2011 to MD 2012. It did 
this on startup:

Simple migration from /Users/whoever//.moneydance to 
/Users/whoever//Library/Application Support/Moneydance
Migration success!

So, I looked things up and I think this folder is a mac thing on where it wants 
programs to keep their files. I think I remember deciding not to care about mac 
preferences when I decided where to store the file on mac and left it using a 
standard java variable so it would work on windows, linux and mac all the same 
way.

MD 2012 migrates or moves the importer setup file maybe the first time you 
install it. You can try this. I created the folder like I said before: 
in /Users/whoever/   create a subfolder:   .moneydance
in there copy /Users/whoever//Library/Application 
Support/Moneydance/mdcsvimporter.props  to  /Users/whoever/.moneydance

It looks like I cannot save new changes to the custom readers but they do show 
up in MD 2012 and hopefully you can use them. 

I don't know if apple made a security change with Java or MD 2012 is doing 
something different but it looks like I will probably have to make a change to 
work with mac's differences.

Original comment by stashu....@gmail.com on 25 Feb 2013 at 4:29

GoogleCodeExporter commented 9 years ago 
I made a fix attempt. Read the description. You can get it here:

http://code.google.com/p/mdcsvimporter/downloads/list
--> test for mac custom readers (reading file) 15.7.8 for MD 2012

unzip it. 
remove your "csv reader" plugin in MD.
restart MD.
select extensions -> manage extensions
then at bottom, select "add from file"
pick the *.mxt file you unzipped.
install it.
test it.

Do you see your readers? Hopefully you can use them again.
You probably cannot save a reader change? another problem for me to look at.

Original comment by stashu....@gmail.com on 25 Feb 2013 at 5:39

GoogleCodeExporter commented 9 years ago 
So there is a folder /Users/daniel/.moneydance with mdcsvimporter.props in it. 

I downloaded your fix but could not install it. After selecting Manage 
Extensions... it downloads the list and then shows extensions to install but 
not "add from file."

Original comment by dcowen...@gmail.com on 2 Mar 2013 at 4:45

GoogleCodeExporter commented 9 years ago 
Oh, and /Users/daniel/.moneydance/mdcsvimporter.props already has a custom 
reader definition in there. 

I am not sure if this is relevant, but I saw that in the latest version there 
is a simple date/description/credit/debit option, so I decided to try it. But 
when I go to select the file, the dialog does not see the file ~/test.csv. So I 
manually type it in, and then all the options are greyed.

Original comment by dcowen...@gmail.com on 2 Mar 2013 at 4:53

GoogleCodeExporter commented 9 years ago 
Hi dcowen,

If the Manage Extensions window does not have an 'Add from File' button, then 
drag-n-drop the 
unzipped mdcsvimporter.mxt file onto the opened Moneydance window.

If you did not purchase Moneydance from the Apple Store, then download the 
latest version, 
currently MD 2012.5 (build 862)
http://infinitekind.com/downloads

HTH -Kevin N.

Original comment by kmnugent...@gmail.com on 2 Mar 2013 at 3:14

GoogleCodeExporter commented 9 years ago 
Hi. If you dont see "add from file" that's a MD question. Try what Kevin said.
After you uninstall mdcsvimporter , restart, and install my 15.7.8 version, 
that one will not look in /Users/daniel/.moneydance but the .../Library/... 
location that Apple uses. So if you moved the md...props file, move it back to 
there, then 15.7.8 should find it and your readers.  It just cannot save to it 
probably. I dont know why.

Original comment by stashu....@gmail.com on 2 Mar 2013 at 10:36

GoogleCodeExporter commented 9 years ago 
Okay, so I figured out  how to install extensions manually, thanks to Kevin. 
Using both methods to install v. 15.7.8 I get this:

"There was an error loading the extension file: java.lang.NullPointerException."

I installed v16 via the gui, and still it does not recognize my CSV file. I 
must be doing something wrong with the CSV file. Can anyone upload a CSV file 
that works?

Original comment by dcowen...@gmail.com on 5 Mar 2013 at 6:36

GoogleCodeExporter commented 9 years ago 
If the readers are showing up properly, that's the main point.
See above my issue-22.doc and here are a couple of issue22 csv files I must 
have tested with before (attached). Mac has a different end of line character.

Original comment by stashu....@gmail.com on 5 Mar 2013 at 1:30

Attachments:

GoogleCodeExporter commented 9 years ago 
Hi. Did you ever test the Mac test version I put out for download?
Does it make your readers show up? Can you save a change to them or just read 
them?

Original comment by stashu....@gmail.com on 27 Mar 2013 at 2:04

GoogleCodeExporter commented 9 years ago 
Sorry for the silence. I have been busy and traveling. 

Yes, I did check the test version (what is currently installed is beta 15.7.7), 
and when I last tried it did not work. But today I tried again, and I can make 
changes and add new custom readers. 

The problem is in actually importing. I get an error stating that the file 
format is wrong. But I noticed that no file reader format is selected when I 
select a csv file to import.

Original comment by dcowen...@gmail.com on 29 Mar 2013 at 8:03

GoogleCodeExporter commented 9 years ago 
Hi. If you are using a mac you need a newer version of the importer like I 
said. Last night I put another new version out which might even work with your 
S $123 amounts.
You can download version 15.7.10 (newest) and let me know how that goes if you 
want.

Original comment by stashu....@gmail.com on 18 Jun 2013 at 4:17

GoogleCodeExporter commented 9 years ago 
I reinstalled the latest version (v. 16, from within Moneydance), and I get an 
error. See the attached screenshots.

Original comment by dcowen...@gmail.com on 22 Jun 2013 at 2:56

Attachments:

GoogleCodeExporter commented 9 years ago 
automatic moneydance version is 3 versions behind. if people tell me 15.7.10 
works in a little while I will put it in moneydance. As for now, you have to 
download the newest from here and install it manually. It's easy. I would go to 
'Extensions' -> manage extensions -> and remove CSV Importer. Restart 
moneydance.
download the newest version from here.
Installation (from home page)
    Download the latest plugin from the Downloads tab above.
    Unzip to any folder if you get a zip file. Otherwise rename a straight mxt file to plain mdcsvimporter.mxt to use.
    Open Moneydance, open Extensions menu, and choose Add....
    Select From File... and click Next.
    Select the .mxt file you just unzipped and click Next.
    Click Yes to allow the plug-in to be loaded.
    Inspect the plug-in information and click Finish. 

You need the newest to work with non US money symbols and reverse , . and 
because you are using a MAC

After that it will probably work. If not go to 'Help' -> 'Console Window' and 
look at where it says your reader would not work.
You could send me that info if you want, but it will continue your transaction 
info.
Hopefully the newer version will just work.

Original comment by stashu....@gmail.com on 22 Jun 2013 at 3:08

GoogleCodeExporter commented 9 years ago 
Yes, right. I downloaded mdcsvimporter-beta-15.7.10.zip and unzipped it and 
tried to install from file. I got the following message: "There was an error 
loading the extension file: java.lang.NullPointerException"

Original comment by dcowen...@gmail.com on 22 Jun 2013 at 3:17

GoogleCodeExporter commented 9 years ago 
Do you want the console output?

Original comment by dcowen...@gmail.com on 22 Jun 2013 at 3:18

GoogleCodeExporter commented 9 years ago 
it has to install extension first.
remove extension.
close MD
start MD
manage extensions, add from file (*.mxt)
use csv importer

Original comment by stashu....@gmail.com on 22 Jun 2013 at 3:24

GoogleCodeExporter commented 9 years ago 
That is what I did (according to your previous instructions), but I checked 
again, and there was no extension installed. I manually deleted a file named 
*.props or something like that from ~/.moneydance, but that did not change the 
situation. I still cannot install the extension.

Original comment by dcowen...@gmail.com on 22 Jun 2013 at 3:44

GoogleCodeExporter commented 9 years ago 
[deleted comment]
GoogleCodeExporter commented 9 years ago 
I did not say anything about deleting the *props* file. 
I just downloaded it and did this myself on my wife's mac for MD 2012.5 (862) 
and it installed fine. 
The extension says "Import File" and it comes up.
Are you doing anything different with java on mac like some sandbox thing? What 
version of java you using? I have:
bash-3.2$ java -version
java version "1.6.0_45"
Java(TM) SE Runtime Environment (build 1.6.0_45-b06-451-10M4406)
Java HotSpot(TM) 64-Bit Server VM (build 20.45-b01-451, mixed mode)

You might want to get help from money dance if the install extension does not 
work for you.

Original comment by stashu....@gmail.com on 22 Jun 2013 at 11:53

GoogleCodeExporter commented 9 years ago 
This morning a new Java version was available, so I updated and retried, and it 
installed. I created a custom file reader and successfully imported data that 
included S$. Thanks for your work on this. 

However, I am back to the problem that custom readers are not persistent. After 
I closed the dialog, deleted the test transaction and reopened the Import File 
dialog, the custom reader was gone. 

My Java version is: 
java version "1.6.0_51"
Java(TM) SE Runtime Environment (build 1.6.0_51-b11-457-11M4509)
Java HotSpot(TM) 64-Bit Server VM (build 20.51-b01-457, mixed mode)

I am not sure what this affects, but I believe that Java operates in a sandbox 
mode in OSX Lion.

Original comment by dcowen...@gmail.com on 24 Jun 2013 at 1:06

GoogleCodeExporter commented 9 years ago 
add new version 15.7.11 
Adds: Can import Categories, added try to find Mac Prefs in more places - has 
all latest

Original comment by stashu....@gmail.com on 12 Jul 2013 at 5:42

GoogleCodeExporter commented 9 years ago 
Thanks, I updated the extension, but still custom readers are not persistent.

Original comment by teconsul...@gmail.com on 12 Jul 2013 at 7:17

GoogleCodeExporter commented 9 years ago 
Ok. I look in 4 places trying to find the importer prefs file.
Would you mind looking for this file and telling me where it is so I can add it 
to the list?

Here's where I look now:

File moneydanceHome = new File( System.getProperty( "user.home" ) + 
"/Library/Application Support/Moneydance", "mdcsvimporter.props" );

moneydanceHome = new File( System.getProperty( "user.home" ) + 
"/Library/Preferences/Moneydance", "mdcsvimporter.props" );

moneydanceHome = new File( "/Library/Preferences/Moneydance", 
"mdcsvimporter.props" );

moneydanceHome = new File( System.getProperty( "user.home" ) + 
"/Library/Moneydance", "mdcsvimporter.props" );

Instead of the final "Moneydance" folder, it could be somplace like:
com.moneydance.apps.md.controller.Main
or com.moneydance

but I only have my wife's mac to try it on. I just tried it on hers and I see 
that it finds the existing file, but when I make a change it does not save it.
I can change the file myself and it saves so I do not know what the problem is 
at this point. 

Can you open the 'console' in moneydance? I cannot and that tells me what is 
happening.

P.S I am closing this issue. For the reader settings not saving, please look at 
issue #25

Original comment by stashu....@gmail.com on 13 Jul 2013 at 3:57