coraallensavietta
commented
6 years ago yes, decided to do this. Need to rewrite small parts of RunSimulation to do this.
Main Questions for this Experiment
increase gaps in parameters until they're trivial. decrease until broken/impossible.
think of this as an experiment. what experimental conditions do we need to run to get a sense of behavior?
think about orders of magnitude to start (want low, medium, high)
Simulations TODO
Ranking Function Parameters:
- losses (1 vs 2)
- parameter scale (probability, rank, logit scale)
- weights (epsilon from .2 to 0.01)
Data Parameters
- variation in N
- even gaps
- uneven gaps #qbeta(1:N/(N+1), 1, 1). Vary the last two parameters for variable gap size
- p c() look at beta distributions for alpha and beta. could crowd them up at one edge. will be symmetric
for below, think about how big average n should be to get p in CI n min: c(50) explore this to see . what differences in p1 and p2 coudl you find with this n? do a power calculation basically n max: se = sqrt(pq/n). think about how small this would have to be to think about how big n needs to be se = sqrt((.1.9)/20) #graph this out to see. If the ranking is always right, then n is too big. If ranks are uniform/terrible, then n too small. n assignment method
could i make this more efficient by running PostSamples only once per simulation? This seems to be the most time-consuming step.