boozallen / argo

Other
44 stars 19 forks source link

Resimulation does not produce new results #5

Closed todd-skinner closed 7 years ago

todd-skinner commented 8 years ago

I'm copying an excerpt of Blake Anderson's email to me describing this bug. See attachment.

During our troubleshooting session, Ryan and I experience several strange things. The cells in question are E10 (producing “0”) and F10 (producing “#DIV/0). I think this is an official bug.

Note E10 and F10 contain op formulas to calculate the mean and standard deviations of a result Argo distribution.

OS: Windows 7 System Info: Excel 2013 Argo Version: 3.0.0

Observation: Given the file as-is, hitting simulate recalculates the workbook and places values in the cells in question. However, re-simulating again does not change those cell values.

Observation: Any change to the options of the simulation (random seed, native excel, number of trials, or latin hypercube sampling) produces values in the cells in question. However, re-simulating again does not change those cell values.

Observation: Using the OP function to extract the individual trial results works. However, re-simulating again does not change the trials.

Conclusion: It would seem to be 1 of 2 things. 1) Could be that the random number generator’s seed is locked accidentally (unlikely) 2) A trigger that calls for a recalculation isn’t firing properly and does not re-calculate the workbook unless absolutely necessary (simulation options are changed or values themselves are changed)

blakeboswell commented 8 years ago

Some of the things described above are actually features (which I'll detail below), but first to address the issue: E10 producing 0, and F10 producing #DIV/0.

Issue

Upon opening this workbook in Excel 2010, E10 produced 0 and F10 produced #DIV/0. After running a simulation the values updated to E10 ~ 10.23 and F10 ~ 2.85. Saving and reopening the workbook resulted in E10 ~ 10.23 and F10 ~ 2.85.

Does the series of outcomes described above replicate the behavior? Or is it that E10 = 0 and F1 = #DIV/0 always? If its the latter case then there is a bug unrelated to the observations.

Argo Simulation Process (Features)

The below should be documented in Argo help material after review. Also, these are referred to as features because breaking up the simulation workload in this manner is what enables faster run times and dynamic simulations

Distributions

Argo distributions have 3 distinct components. Sampling, Inverse CDF, Correlation.

The above are independent of running a simulation. What this means in practice:

Simulation

The simulation process translates an Excel workbook into a function, SF, that includes distributions as arguments, i.e. SF(CDF(U, ...), ...)

If a simulation is rerun then a new SF(CDF(U, ...), ...) is constructed. Note that U is not updated. If no other aspect of SF was changed then the exact same SF will be constructed.

If a formula is changed, then clicking simulation will generate a new SF that is constructed differently to reflect the formula change.

Observations

Taking the above into account, the three observations are the expected behavior.