carykh / PrisonersDilemmaTournament

Watch This Place's awesome video about iterated Prisoner's Dilemma for context! https://www.youtube.com/watch?v=BOvAbjfJ0x0
MIT License
205 stars 160 forks source link

Output head-to-head results in a headToHead.csv file for quick comparison #31

Open l4vr0v opened 3 years ago

l4vr0v commented 3 years ago

As I test new strategies, parsing results.txt by hand gets rather frustrating. For example, I am looking at two of my strategies- nprtt and ultimateDetective- to understand why nprtt is marginally outperforming ultimateDetective, and this is basically impossible with results.txt. With headToHead.csv, I can just import the file into Excel or Google Sheets and see which strategies nprtt does better on than ultimateDetective (and vice versa) so I know what to dive deeper into, in results.txt.

This Pull Request also does a couple other quality of life improvements:

You can verify yourself that headToHead.csv is properly parsed by Excel and Google Sheets.

l4vr0v commented 3 years ago

To follow up on the use case mentioned in the first comment, here's an actual head to head sheet and this made it clear to me that uftft was what's breaking my ultimateDetective and causing it to perform unexpectedly worse than nprtt (which makes sense since ultimateDetective tries to detect and exploit a ftft and with a uftft it will instead wind up thinking it's alwaysCooperate, spamming defections, and scoring low).

This feature has significant utility because it made a previously annoying operation really fast.

l4vr0v commented 3 years ago

This has some similarities with #30 but the CSV output in #30 is a different format (albeit a more compact one) that would require a formula to generate a sheet like this. It may make sense to merge both. Or you could merge in #30, close this one, and I can share a spreadsheet template that converts the output of #30 into a head-to-head table like the one generated by this PR.