Comparison of running scripts with stationary and non-stationary models in the background

[alexlafleur]

The goal was to run the same script of evaluation with different models in the background (the normal stationary markov model, and the non-stationary markov model with delta parameter). A delta parameter of 0 should yield the same results as the stationary model.

[alexlafleur]

Performance using MMFamily1

Performance using QMMFamily1 and delta=0

Error using MMFamily1

Error using QMMFamily1 and delta=0

[alexlafleur]

Plots without reusing the mmf1_0 object from sampling the qmm

[greenTara]

OK, the behavior of the heatmap does make sense due to the increase in shift and window size with increasing taumeta along a row.

[greenTara]

To close this issue, we need to make a final set of calculations for mm and qmm with delta = 0, to verify that they are still producing the same thing, after the refactoring of the code. It is not clear if we can get exactly the same model and simulation by setting the random seed. If that would work, this could be used as a unit test, where we would compare the simulated data in each case, rather than perform any estimations. Otherwise, we need to look at the statistical results, (averaged over large num_trajectories and num_runs) in order to make a comparison.

[alexlafleur]

We do have the same Spectral MMs in both models, but the simulation does return different arrays for both models even though I set a seed.

If we go with the other possibility, what values do we assign num_trajectories and num_runs?

[greenTara]

Let's use numsims = 256.

num_runs = 1 (with verification that the transition matrix is the same for MM and QMM)

[alexlafleur]

What about num_trajs?