Closed itsdfish closed 3 years ago
I'll have a look. For a long time I had on my todo list to check the start argument. Would you like to read multiple csv files.
I needed two read multiple files for the WAIC/LOO tests. However, it might be nice to have to option to pass an array of file names or a single file name. I think the easiest way to achieve both use cases would be to dispatch on a String
and wrap it in a vector:
read_csv_files(file_name::String, n_samples, output_format=:namedtuple;
include_internals=false, kwargs...) = read_csv_files([file_name], n_samples, output_format=:namedtuple;
include_internals=false, kwargs...)
One advantage of this approach is that it would not require modifying your original function to accomodate a case for reading a single file.
Personally, I wish Stan would save its metadata in a separate file. So that we can load chains easily. I'm sure you have a similar opinion.
Yes, that is a wish that's been around for a long, long time. I haven't really looked at the JSON format.
Do you mean n_samples or n_chains in above signature? :namedtuple is the default output_format and it organizes the parameters such as a.1, a.2, etc. in arrays.
I passed n_samples to determine the number of rows and used the length of file_names to infer the number of chains.
I've added a function (not exported) StanSample.read_csv()
to StanSample.jl. Basically you provide a base path, n_chains and n_samples (for each chain) and the output_format (by default :namedtuple). But there are many roads that lead to Rome, yours in the test script also works.
Thanks, Rob. I think this will be useful for others as well. My approach to reading the files was very inelegant: I manually removed Stan's metadata and used CSV
to read in the files. Although it works, it is not ideal if we need to change the test. I will update the test accordingly later.
Hi Chris, I definitely need to do more testing and also check if the start
argument works. Once convinced, I prefer to use this code in the current read_csv_files as well. And I have also an option on my list to read selected chains, e.g. 1,3 and 4.
Will also take a closer look at what tests we have in place now and how your test results compare with R's loo.
Hi Rob,
I wanted to use this package to read Stan output files for the LOO and WAIC tests. Unfortunately, this required a model object. It might be useful to have two methods: one based on the model object, for convenience, and a generic method that does not require any special object.
I tried to split the methods up, but ran into some issues that I could not resolve easily. Here is what I started on. The function
convert_a3d
threw an error because I passedstart=1001
.