search

cangiuli / hanabi

2 stars 1 forks source link

Run multiple games #4

Closed fpvandoorn closed 8 years ago

fpvandoorn commented 8 years ago

To compare performance of a new feature, it would be nice to run multiple games and see if the average score increases. If you run multiple games, you probably don't want the game log, but only the achieved score (or maybe only the average score of all games). It would be nice if we can just add a flag to the terminal command to specify how much games we want to play (by default 1).

cangiuli commented 8 years ago

I would like to hold off on having command-line options at the moment, but I can definitely add a function to HANABI that runs many iterations of the game without printing output.

What is the most useful interface? int -> int returning the average score? Or would you prefer int -> int list returning all scores?

fpvandoorn commented 8 years ago

Probably int -> int list is nicest, so that we can do more statistics on it than just the average. In the Python implementation we printed the average, standard deviation of the average, percent of the time it reached maximal score, and the standard deviation of that percentage. We don't need all of those immediately, but if the program is doing well, it's easier to look at how often it gets 30, than looking at whether its average score is 29.78 or 29.79.

fpvandoorn commented 8 years ago

Nice!