getguesstimate / guesstimate-app

Create Fermi Estimates and Perform Monte Carlo Estimates
http://getguesstimate.com/
MIT License
2.34k stars 125 forks source link

Sensitivity analysis varies depending on on ordering of sample #1051

Open PeterFavaloro opened 8 months ago

PeterFavaloro commented 8 months ago

I'm using a "custom" distribution by pasting in my own data from a spreadsheet. In one case I sorted the values (smallest to largest) before pasting in to Guesstimate; in the second case I randomized the order of the values. Note the differences in the sensitivity analysis graph. It seems like differences like this should not result from randomizing the order of the custom data, or else the documentation should warn users that the order of their data matters a lot.

Screen Shot 2024-03-13 at 5 16 00 PM Screen Shot 2024-03-13 at 5 16 34 PM

In other cases I get differences in the reported r2 of the sensitivity analysis, and in the reported median of the dataset, just by sorting the values. I think what's happening in those other cases is that I pasted in more than 1000 values, and Guesstimate truncates to just the first 1000 before some analyses.