carykh / PrisonersDilemmaTournament

Watch This Place's awesome video about iterated Prisoner's Dilemma for context! https://www.youtube.com/watch?v=BOvAbjfJ0x0
MIT License
205 stars 160 forks source link

Analysis and performance #75

Open EFHIII opened 3 years ago

EFHIII commented 3 years ago

I've spent more time trying to put together tools for analysis and improving performance than I have working on my strategies... oh well.

I figured I'd at least share: https://github.com/EFHIII/PrisonersDilemmaTournamentTools

Regarding performance, I tried to do a few things including creating a cache, looking for 3-3/0-5 patterns for early escape, looking for statistically consistent matches for early escape, and multi-threading.

There's probably some things that could be improved, but it works pretty well; especially if you're constantly modifying one file and re-loading it, which lets you skip having to calculate the majority of the pairings making it take seconds instead of minutes.

For analysis, besides just looking at the matches between 2 strategies, I've written a script that compare sets of strategies against others and a script that lets you look at what the standings would look like with different amounts of certain strategies in the meta. For example, simulating 1e100 randoms via

node meta.js random=1e100

This takes almost no time to run because it uses the cache file, although it's not perfectly accurate since the cache (by default) is only built running each strategy at most 100 times per pairing. This can be increased if desired, for example:

run n=1000 --delete-cache

I might add some more tools & features later.