search

endolith / elsim

Election Simulator 3000: Simulates a variety of elections and voting methods
https://endolith.github.io/elsim
MIT License
7 stars 4 forks source link

Add uncertainty to estimates #13

Open endolith opened 2 years ago

endolith commented 2 years ago

"Estimated variance of estimated probability"

https://gist.github.com/endolith/9e73ea30511f0befb9b336526f34d0a6

https://stats.stackexchange.com/q/430409/11633

https://stats.stackexchange.com/q/576491/11633

endolith commented 2 years ago

Error bars often represent one standard deviation of uncertainty, one standard error, or a particular confidence interval (e.g., a 95% interval). These quantities are not the same and so the measure selected should be stated explicitly in the graph or supporting text.

🤔

endolith commented 2 years ago

https://matplotlib.org/stable/api/_as_gen/matplotlib.pyplot.errorbar.html

endolith commented 2 years ago

Figured out how to do 95% confidence intervals and plot error bars for binary tests like Condorcet cycle likelihood and Condorcet efficiency using scipy.stats.binomtest(n_successes, n_trials).proportion_ci():

https://github.com/endolith/elsim/compare/error_bars

Adding confidence intervals for total utility or Social Utility Efficiency, on the other hand, is much less obvious and I've spent far too long trying to figure it out. scipy.stats.bootstrap is probably appropriate, but very slow. Generating more data could provide credible intervals(?) but is also extremely slow. Giving up for now.