search

slew77 / hle-ofx-quotes

Automatically exported from code.google.com/p/hle-ofx-quotes
0 stars 1 forks source link

sunriise-Build_20110628_19-app unable to open money 2002 UK version #35

Open GoogleCodeExporter opened 8 years ago

GoogleCodeExporter commented 8 years ago 
What steps will reproduce the problem?
1. attempt to open attached money file (NO PASSWORD)
2.
3.

What is the expected output? What do you see instead?

What version of the product are you using? On what operating system?
sunriise-Build_20110628_19-app 
windows 7 Ultimate 64bit

Please provide any additional information below.

Original issue reported on code.google.com by progers...@hotmail.com on 6 Jul 2011 at 8:24

Attachments:

GoogleCodeExporter commented 8 years ago 
Will take a look.

Original comment by hle...@gmail.com on 7 Jul 2011 at 12:48

GoogleCodeExporter commented 8 years ago 
I think I have a fix for being able to open the 2002 file. I also want to take 
a look at the off by 100x issue. How did you create your copy of quotes.csv. 
Was it created by hleOfxQuote tool? Can you attached a sample quotes.csv 

Thanks,

Original comment by hle...@gmail.com on 7 Jul 2011 at 1:52

GoogleCodeExporter commented 8 years ago

Original comment by hle...@gmail.com on 7 Jul 2011 at 1:53

GoogleCodeExporter commented 8 years ago 
Hi,

the quotes.csv was created using hleOfxQuotes-Build_20110628_19 and using the 
ft.com import csv file, when I import hleOfxImport.csv the quotes.csv is 
created.

I have attached the import file (hleOfxImport.csv) and the generated quotes.csv

It would be nice if the imported quotes appeared in money as online rather than 
a buy,  I assume they appear as a buy because the  previous line copied was a 
buy?

Regards

Paul

Original comment by progers...@hotmail.com on 7 Jul 2011 at 8:46

Attachments:

GoogleCodeExporter commented 8 years ago 
Cool. Got the test *.csv files.

I've fixed hleOfxQuotes to convert incoming price with currency GBX to GBP 
(price * 100.00).

Regarding "imported quotes appeared in money as online rather than a buy", I 
think you are on the right track but I could not work out which column to 
change to make it an update instead of a buy. I will make available a new 
version with the following fixes:
  1. Can open 2002 file.
  2. Auto-convert GBX to GBP
Can you do a quick test:
  1. quotes.csv has converted price from GBX -> GBP
  2. 2002 file can be opened in both read-only and write
  3. prices can be updated via command-line tool

Also it will be great if you can "explorer" the SP table and which column can 
be change to make quote "online" rather than "buy".

I will add another comment with URL's to new version.

Best,

Original comment by hle...@gmail.com on 7 Jul 2011 at 10:02

GoogleCodeExporter commented 8 years ago 
URL to new version: 
http://sunriise.sourceforge.net/out/hleofxquotes/Build_20110706_31/

Original comment by hle...@gmail.com on 7 Jul 2011 at 10:20

GoogleCodeExporter commented 8 years ago 
[deleted comment]
GoogleCodeExporter commented 8 years ago 
Paul, thanks for your help. I need clarification:
  . "all the downloaded rows had a false entry" which column is that? The one has false entry?

From what I can gather from your finding, I think I just need to look for a 
previous line with src == 6 to duplicate from. You think that will be 
sufficient?

Best,

Original comment by hle...@gmail.com on 8 Jul 2011 at 3:45

GoogleCodeExporter commented 8 years ago 
Re: sorting rows by clicking column header

I've added issue: http://code.google.com/p/hle-ofx-quotes/issues/detail?id=36

Original comment by hle...@gmail.com on 8 Jul 2011 at 5:03

GoogleCodeExporter commented 8 years ago 
[deleted comment]
GoogleCodeExporter commented 8 years ago 
I don't think that will work as you may have purchased new stock in which case 
src == 1 so it will not find an entry to update.

I have done some more research on this please look at my findings below and let 
me know what you think:-

I have observed what  occurs during an online update, have a look at the 
attached spreadsheet where I have put in the rows for the stock update 
(National Grid Stock Symbol NG.) before and after the update. (Sheet SP-rows 
during online updates)

7th July  the first online update:-
Row 2 (hsp62) shows fFractPrice condition TRUE

8th July  second online update:-
Row 9 (hsp114) shows fFractPrice condition TRUE
However yseterdays row has now been changed
Row 12 (hsp62) shows fFractPrice condition FALSE

I have used sunriise-Build_20110706_31-app to changed the condition of (hsp114) 
fFractPrice from TRUE to  FALSE and in Money, the online update for today is 
still present

Based on the behaviour above and to keep your code straight forward I would 
continue as you have been doing and after duplicating the stock  row found in 
addition to changing the date (dt)  and  price (dPrice) I would:-

change if it isn't already that value:-
src=6, fFractPrice=FALSE, dStrike="", fFractStrike=FALSE, dOpen="", 
fFractOpen=FALSE, dHigh="", fFractHigh=FALSE, dLow="", fFractLow=FALSE, dPE="", 
fFractPE=FALSE, dtExpire="", fClosing=FALSE, vol="", hss="", dChange="", 
fFractChange=FALSE, dtSerial=The actual time date time of this entry NOT the 
date time of the share price being imported as this could be different.

by doing this the row only contains information that has been imported for that 
date. The method above assumes that there is only one import a day which I 
would recommend is performed after the market has closed.

AS an FYI
I also ran a second onlineupdate on the 8th of July and money updated row 
(hsp114) with the current stock price (dprice)  and date time (dtSerial ) which 
had changed as opposed to creating a new line so this explains the use of the 
flag fFractPrice, i.e.
 if the  date of the update is the same as the latest entry for the stock and fFractPrice  is TRUE then update the row, if the date is newer then the last entry set fFractPrice to False and insert a new row.

Based on this observation then I would recommend that you place a check in your 
code to prevent an  update occurring  if the quote has already been updated for 
that date as I am not sure how msmoney will react with multiple rows with the 
same date, unless you update the code to use the fFractPrice flag as above. 
(now there's a challenge or possible enhancement?  :-)  )

Hope this helps
regards

Paul

Original comment by progers...@hotmail.com on 8 Jul 2011 at 12:29

Attachments:

GoogleCodeExporter commented 8 years ago 
Hi Paul, thanks for the detail data. I appreciate it.

I will go over them more closely when I have some spare time this week-end.

Best,

Original comment by hle...@gmail.com on 8 Jul 2011 at 2:34

GoogleCodeExporter commented 8 years ago 
Hi Again,

I have been reviewing the data again and I would like to revise the use of the 
fFractPrice flag.
I now believe that this indicates that the dPrice is fluctuating as it would 
during trading hours. dt remains the same during online updates, dtSerial 
updates with the current date/time stamp.

when money runs online update it checks the actual date, if this is the same 
day as dt, it updates dPrice with the current quote and dtSerial with the 
current date/time stamp
if the date of the update is different i.e. a different DDMMYYY then it checks 
fFractPrice 

if fFractPrice is TRUE then it downloads the closing price of the stock for the 
date in dt and the sets fFractPrice to FALSE and updates dtSerial with the 
current date time stamp.

if fFractPrice is FALSE then it creates a new row and populates it with the 
current stock price  and sets fFractPrice to TRUE 

This way the closing price on the day is always correct in money.

This doesn't change what I recommended earlier but reinforces the fact that the 
update information should be downloaded after the markets have closed to get an 
accurate history

Have a good weekend

Regards

Paul

Original comment by progers...@hotmail.com on 8 Jul 2011 at 6:24

GoogleCodeExporter commented 8 years ago 
Hi Paul, I've read through your posts. Thanks a lot for the detailed data.

  Let get some easier stuff out of the way first. 
  * I believe the usage of 'src':
    . 1 is buy
    . 5 is manual update (user entered the price him/herself)
    . 6 is online update
  * Base on your analysis, fFractPrice indicates non-closing price. I think it is only meaningful for Money online update. Another words, a price where fFractPrice == true is still being treated as a valid last price.

  Next easy case, if there is existing entry with today date and src == 5 or 6, then we can just update the price and dtserial for that entry and we are DONE.

  The hard case is when there is NO today entry that we can just update price (and dtserial). How to create a new entry?

  . I had proposed to clone of last entry with src == 6. You had concern: "I don't think that will work as you may have purchased new stock in which case src == 1 so it will not find an entry to update.". I meant to say look for an existing entry in the past with src == 6 (does not have to be the exact previous entry n - 1). As long as there is an entry of src == 6 in the we can just clone it, set price, dt and dtserial. (I would amend this to include src == 6 or src == 5)

  That left the case where there is no previous entry with src == {5, 6}. In that case, look for src == 1. Then set (as you recommend): 

src=6, fFractPrice=FALSE, dStrike="", fFractStrike=FALSE, dOpen="", 
fFractOpen=FALSE, dHigh="", fFractHigh=FALSE, dLow="", fFractLow=FALSE, dPE="", 
fFractPE=FALSE, dtExpire="", fClosing=FALSE, vol="", hss="", dChange="", 
fFractChange=FALSE, dtSerial=The actual time date time of this entry NOT the 
date time of the share price being imported as this could be different.

Sorry if it sounds like I am repeating of most what you are saying. There were 
a lot of data and I just want to make sure I can re-iterate them in my own 
words.

Best,

Original comment by hle...@gmail.com on 11 Jul 2011 at 11:48

GoogleCodeExporter commented 8 years ago 
Hello,

No problem restating this in your own words, I do the same, it helps me order 
my thoughts.

The money online quote download appears to have information that it does not 
use such as low and high price, volume etc. either that or I have not found 
where it presents this information in money which is possible as I only use the 
functions that I need.

As a precaution when it comes to updating/cloning a previous line src == 6 or 
src == 5  for a new date, I would clear out any existing quote data that has 
not been downloaded.

Let me know if I can help in any way

Regards

Paul

Original comment by progers...@hotmail.com on 12 Jul 2011 at 12:53

GoogleCodeExporter commented 8 years ago 
Hello there,

I was just curious if you had made any progress with the changes?

I have collected prices for the last 2 months so that I can give your changes 
some rigorous testing :-) 

regards

Paul

Original comment by progers...@hotmail.com on 5 Sep 2011 at 3:22

GoogleCodeExporter commented 8 years ago 
I have not spent much time recently in this area though I believe the current 
codes is in a state that can be tested: I've implemented it as much as I have 
planned to.

Let me refresh myself in the current code so far and will give an update on 
what the current state is.

Thanks for the reminder.

Original comment by hle...@gmail.com on 5 Sep 2011 at 9:48

GoogleCodeExporter commented 8 years ago 
No problem,

more than happy to test and provide results.

Original comment by progers...@hotmail.com on 6 Sep 2011 at 12:22

GoogleCodeExporter commented 8 years ago 
Paul,
  Can you give it a try.
  . Download: https://repository-hleofxquotes.forge.cloudbees.com/snapshot/com/le/sunriise/sunriise/0.0.2-SNAPSHOT/sunriise-0.0.2-20110908.022801-19-jar-with-dependencies.jar
   . You can rename sunriise-0.0.2-20110908.022801-19-jar-with-dependencies.jar to something shorter if you like. For example: sr-new.jar
   . Now run
java -cp sunriise-0.0.2-20110908.022801-19-jar-with-dependencies.jar 
app.UpdateStockPrices quotes.csv sample.mny 
      . argument #1: quote file
      . argument #2: *.mny file
      . argument #3 (optional): password

There should be some log output saying what the tool tries to do: looking for 
entries with src = ...

  Let me know what you find. Thanks for volunteer to test. I appreciate it. Sorry I have not been able to spend much time on this recently.

Original comment by hle...@gmail.com on 9 Sep 2011 at 6:41

GoogleCodeExporter commented 8 years ago 
I initially tried the new code on a simple test file and all appeared good, I 
see that the new entries created are with SRC==5 (update) as opposed to SRC==6 
Online; if the entry is online then in the Home Page/Investment Summary 
reflects the change in price whereas if it is an update then the change shows 
as 0.00 (Just an observation)

I next ran the new code on a copy of my current money file and started 
importing a series of stock updates working back in date; this appeared to be 
going ok until the 18 aug where I get an IOException after importing a couple 
of entries I ran money Salvage prior to this and it reported no broken links 
and ran this after the error and it again reported no broken links. I ran the 
code again and got the IOException again. see attached zip file with the log 
file and csv being imported. msmoney has no problems opening the file.

Cheers

Paul

Original comment by progers...@hotmail.com on 12 Sep 2011 at 8:58

Attachments:

GoogleCodeExporter commented 8 years ago 
Thanks. I am taking a look at the IOExeption.

Original comment by hle...@gmail.com on 16 Sep 2011 at 2:42