Hard code in max number of iterations for SMC run?

[gtng92]

Two runs "completed" in issue #123 had to be manually stopped. example-compartmental.beta.gamma.mu.pdf example-compartmental.RF.dist.iter361.pdf

After saving the TSV files and creating the PDFs of the results before stopping the run, it was clear that the runs had the potential to run infinitely. Some of the parameter estimates would flatline at a value for over 100 iterations, or vary within a param value range of 0.5.

[ArtPoon]

Need to decide what is diagnostic of a run that is stuck in an infinite cycle. Is it no change in epsilon after N steps?

• examine different stopping criteria (epsilon, particle composition, acceptance rate)
• come up with a potential stopping criterion and implement
• run a few tests and close -- if we encounter additional problems, re-open and try a different stopping criterion

[ArtPoon]

Use last 10 runs with acceptance rate of 0 as a stopping criterion (failure), notify user that the prior distributions may be misconfigured.

[ArtPoon]

smc.config contains an acceptance rate threshold but it doesn't seem to be implemented correctly because these runs should have been stopped.

[gtng92]

Re-running simulation that produced the PDF using the RF.dist and recording the epsilons and acceptance rates in a separate file. Acceptance rates are fine, but epsilon is freezing at a value.

[ArtPoon]

I examined these logs with @gtng92 and it looks like all perturbed particles (from proposal distributions) are rejected over a stretch of iterations, which results in no change in epsilon. Need to investigate a couple of things:

• RF distance may be too sensitive, do we get the same result on the same data and settings but with a different distance metric? (@gtng92 has already done this exact same analysis with kernel distance, KF dist, path.dist and Sackin and did not observe a freezing epsilon, which suggests that this issue is characteristic of RF dist..)
• Does this still occur if we reduce the breadth of the proposal distributions?

Proposed actions:

• Close investigation of RF.dist.
• Retain current premature stopping criteria:
  1. acceptance.rate has not changed in last 10 iterations
  2. epsilon has not change in last 10 iterations
• Allow user to adjust these criteria with default to 10 iterations