JuliaStats / TimeModels.jl

Modeling time series in Julia
Other
57 stars 28 forks source link

Identify R packages to test against and benchmark against. #9

Closed milktrader closed 10 years ago

milktrader commented 10 years ago

It seems most team members are comfortable with R and it's clearly the standard against which to test and benchmark our functions.

AndreyKolev commented 10 years ago

Currently in GARCH package I have a test which compares model coeficients against those obtained by fGarch model fitted using some historical quotes, I suppose I just need to switch to spx.csv data.

milktrader commented 10 years ago

You can write tests that use the data in the test/data dir if you wish of course, but that's entirely optional. Also, please feel free to put data sets in there that your tests use. I've renamed them unittest.csv (508 rows) and benchmark.csv (15,851 rows). The reason I've chosen these is because it's easy to get into R's environment with the quantmod package.

getSymbols("^GSPC", from='1900-01-01', to='2012-12-31') # benchmark.csv
getSymbols("^GSPC", from='1970-01-01', to='1971-12-31') # spx.csv

R returns objects that have the correct order for dates so that taking returns makes sense. The TimeSeries.readtime function takes care of it for us when reading from the test/data directory.

Have you used the rugarch package? Also, I've used forecast in the past.

AndreyKolev commented 10 years ago

The rugarch is the most powerful garch package, but it uses specification objects to specify models, so it's a little more cumbersome to use than fgarch. Given the currently a very basic functionality of my garch code it's more convinient to test against it for me. As for the test data I've also used S&P 500 but a small subset starting from 2010, so it's better to use your files I suppose.

milktrader commented 10 years ago

Let's use fgarch for the garch tests and vanilla R arima for the arima tests. For the kalman.jl file I'm not sure but we can find a reasonable solution and fold that issue into a less general one that this broad one.