milktrader / Quandl.jl

Julia api to Quandl open source financial, economic and social datasets
Other
67 stars 31 forks source link

Use generic CSV parser #1

Open johnmyleswhite opened 11 years ago

johnmyleswhite commented 11 years ago

We still need to finish integrating a generic CSV parser into Base, but it seems that you'd be better off using a general-purpose parser than doing comma-splitting by hand. There's probably a while before the generic parser works well enough for your use, but I thought it would be good to raise the idea now while you're working on your package.

milktrader commented 11 years ago

Ok, thanks. Currently the data is being parsed through an awkward combination of non-general control flow. It works, but only for one value column and I'm not capturing the column name at present.

milktrader commented 11 years ago

Brain storm

readtable(s::String, time=true)

Where the first column is parsed as a Calendar object. Should I open an issue?

johnmyleswhite commented 11 years ago

If we start to do automatic time series conversion, I think we shouldn't hardcore which columns are time series, but either (a) guess from the inputs or (b) require that a list of column names be specified by the user.

milktrader commented 11 years ago

read.zoo in all its kwarg glory

> args(read.zoo)
function (file, format = "", tz = "", FUN = NULL, regular = FALSE, 
    index.column = 1, drop = TRUE, FUN2 = NULL, split = NULL, 
    aggregate = FALSE, ..., text) 

The format arg is a string to pass to the Calendar.parse method. tz allows one to over-ride the local machine, index.column defaults to the first one for time, which is good for well over 98% of time series data. I'd pick all three for a readtable method.

milktrader commented 10 years ago

The readable method would solve this issue as well as make the code for Quandl a lot neater (and reliable given the trouble with missing values).

Currently, readable does not support http requests but it appears to be stubbed out to do this in the future.