Option to automatically end test upon reaching N sample size?

splitrb / split

:chart_with_upwards_trend: The Rack Based A/B testing framework

https://rubygems.org/gems/split

MIT License

2.7k stars 367 forks source link

Option to automatically end test upon reaching N sample size? #477

Open imweijia opened 7 years ago

imweijia commented 7 years ago

Just wondering if this can be configured as an option whereby the test can be paused when N sample size of participants is reached?

pandathletic commented 6 years ago

Has there been any progress or discussion on this feature? I also think that would be very useful.

robin-phung commented 4 years ago

To revive this discussion, what were peoples thoughts on how assignment would work in situations where the threshold N was reached and individual alternatives do not meet the required sample size to reach statistical significance?

Assuming the algorithm chosen has an element of randomness I imagine a solution would be to define threshold N per alternative and have the experiment end once all alternatives reach the specified minimum.