Closed greenTara closed 7 years ago
Any ideas on that yet?
We need a skript that takes a fixed dataset with fixed set of parameters - only window_size and r are varying: taumeta=4 shift=64 num_trajectories=2 len_trajectory=8^k+16*shift num_estimations = 16 num_states=4
window_size= range(128, 8^k > 4000), increasing by a factor of 2 Take the latest error within each of the window_size varying runs, run this all together 8 times with new samples, and average the errors.
The output should be two point plots showing LOG values:
num_trajectories=8
The first task is to increase the number of runs (num_runs) so that the statistical fluctuations in these plots are smoothed out. Let's put num_trajectory back to 2 - all that is doing is reducing the error of the estimation, and that is not what we care about at this point.
Once we have enough runs so that we see a smooth dependency in these point plots, we need to see if there is an effect from num_estimations. I would like to perform the calculation over a range of num_estimations (1, 2, 4, 8, 16), and plot these as separate curves on the same graphs, with the axes the same as before.
The goal is to optimize the script so that a large number of runs can be made without unnecessary calculation. There are several points to address, some already completed:
As far as I can tell, that is all that must be changed regarding the plots.
For checking the formula, add the following set of print statements, one set for each point on the graph:
Another optimization:
There are a few places where renaming variables will lead to code that is more self-documenting.
That should do it. The number of runs will need to be at least 256, and may need to be more to get stable values for the expectation of the error.
There has been no further comments on this issue for 15 days, but there was I believe some additional work on this script. Is there a related issue?
I have an improved formula for the expectation of the error in the MM case:
err_bayes[k]/err_naive = math.sqrt( (1+math.pow(r,2*k+1))/((1+r) ))
Also
w = self.window_size err_naive = c/math.sqrt(w * num_trajs) where c is a function of eta, scale_window, taumeta, etc.
Discussion continuing on #31
The formula currently used for the effective window size is not completely accurate. There is a small underestimate of the effective window size (for a given value of "r").