search

Axelrod-Python / Axelrod

A research tool for the Iterated Prisoner's Dilemma
http://axelrod.readthedocs.org/
Other
718 stars 263 forks source link

Desired New strategies #379

Open marcharper opened 8 years ago

marcharper commented 8 years ago

Some of these may be implemented under other names already, please ask if you are unsure! Feel free to add any new ones to the list. Note that we are happy to have original contributions as well!

The "invincible strategies" in this paper which can all be implemented as special cases of the MemoryOne or LRPlayer classes.

The two "most abundant" memory one and memory two strategies in this paper.

Adaptor from Simple Adaptive Strategy Wins the Prisoner’s Dilemma second_pdf

Specific strategies evolved in Evolutionary game theory using agent-based methods such as GCA.

Strategy MO and Strategy SO from this paper

Strategies implemented in PRISON (look in classics.str):

and see this paper

From CoopSim:

Many strategies in this paper are not yet in the library:

From "Exploiting Evolutionary Modeling to Prevail in Iterated Prisoner’s Dilemma Tournaments":

From this page (see also the bibliography) for the 20th anniversary tournament:

From here:

From this paper and also here:

Any of the interesting finite state machine strategies in the papers with fortress (and other papers authored by Wendy Ashlock and Daniel Ashlock, and collaborators)

Many from this paper. Note the several are already in the library, including ALLC, ALLD, TFT, WSLS, willing, hopeless, and desperate (and possibly others).

From these two papers:

From this page:

From the mythical tournament preliminary to Axelrod #1:

From this publication:

From this paper:

From this paper:

From this library (if the license is compatible):

Others:

No-tricks Strategies described here

Theory of mind strategies discussed here.

Would be neat to have strategies based on:

Translate Fortran strategies available in https://github.com/Axelrod-Python/axelrod-fortan to python.

drvinceknight commented 8 years ago

This is a brilliant issue: :+1:

drvinceknight commented 8 years ago

This strategy was mentioned on reddit (Random TitForTat and/or Grudger: defects with random probability):

https://www.reddit.com/r/GAMETHEORY/comments/480xb3/how_to_beat_this_strategy_of_prisoners_dilemma/

flammino commented 8 years ago

Would you mind if I started working on a few of these? I'm sure I'll be slow since I'm new to programming and I'm going to school while working full time, but it looks like they've been posted for awhile. If there are any strategies on the list that are particularly easy to implement I'd be happy to start with one of those.

drvinceknight commented 8 years ago

Absolutely @Adam-Flammino: please do!

I suggest that when you decide on a strategy you run it past us just to make sure it hasn't been implemented yet (it's possible that they're on the list but have already been implemented).

It could also be worth checking here: http://axelrod.readthedocs.io/en/latest/reference/all_strategies.html

That's the list of the strategies that are definitely in the library :)

As far as a suggestion for an initial one to go for, perhaps pick "Nasty Tit For Tat" which is described as "tit-for-tat but attempts to exploit non-retaliatory strategies by playing a DD with some probability". If you think there's another one in the list that you like the look of please do go for that though :)

Also: if you need help you can always pop in to this chat room https://gitter.im/Axelrod-Python/Axelrod there are usually a few of us around that can help :)

flammino commented 8 years ago

That actually sounds a little similar to how I play risk. I've definitely got some more reading to do before I start, but I'd be happy to try. Thanks for the chat link too!

marcharper commented 8 years ago

@Adam-Flammino We're happy to e.g. help write tests, just open a PR or ask us here (or on gitter).

flammino commented 7 years ago

@marcharper thanks! Haven't gotten to start yet, but I appreciate the help! Hopefully before too long I'll be able to actually help out with the project- it looks interesting

flammino commented 7 years ago

So I got to actually sit down and look at things today. I appreciate all the variations of tit for tat being in the same .py file- makes it easier for someone who's still learning like me to see the right format. Another quick newb question- should I submit a new .py file with my pull request or just add my code to the end of the existing file?

marcharper commented 7 years ago

@Adam-Flammino Either way, really. We don't have a scheme for which strategies go in which files, we just group them as we go wherever they seem to fit.

flammino commented 7 years ago

@marcharper Cool. Thanks again.

drvinceknight commented 7 years ago

If you're going to go with nasty tit for that as the strategy I'd suggest that you just add that to the bottom of the existing tit for tat file:)

flammino commented 7 years ago

I appreciate how welcoming you all have been and I do think this is a very interesting project I'd like to help with eventually, but some life changes just came up and I don't know when I'll have time to actually contribute to this. I plan on coming back (hopefully with enough experience to make real contributions), but I'm not sure when that will be so don't hold any of these for me.

meatballs commented 7 years ago

@Adam-Flammino No problem - thanks for the contributions so far!

We'll still be here when you have time again. All the best.

souravsingh commented 7 years ago

I would like to help in writing some of the strategies.

meatballs commented 7 years ago

@souravsingh Are you here at PyCon UK?

souravsingh commented 7 years ago

@meatballs I couldn't come to PyCon UK due to visa issues. But I can try and help remotely.

drvinceknight commented 7 years ago

Welcome @souravsingh! Drop in to our gitter channel: gitter.im/Axelrod-Python/Axelrod

Otherwise, take a look at the contribution guidelines and don't hesitate to ask any questions :)

http://axelrod.readthedocs.io/en/latest/tutorials/contributing/strategy/index.html

mturzanska commented 7 years ago

Hi. Is soft_tf2t up for grabs? gradual_killer needs crossing out because it's already implemented.

drvinceknight commented 7 years ago

@mturzanska I'm afraid that that's in already to under TitFor2Tats: http://axelrod.readthedocs.io/en/latest/_modules/axelrod/strategies/titfortat.html#TitFor2Tats

(Please correct me if you think I'm not quite right reading the two logics).

mturzanska commented 7 years ago

You're right. Then continuing from the top:

Please let me know if I sorted it out right this time. Would you mind if I start working on those?

drvinceknight commented 7 years ago

Would you mind if I start working on those?

That would be awesome! Let us know if we can assist in any way :) 👍

souravsingh commented 7 years ago

Are Resurrection and DoubleResurrection strategies implemented? If not, I would like to work on them.

marcharper commented 7 years ago

I don't believe so, what reference are they from?

souravsingh commented 7 years ago

They are from CoopSim

marcharper commented 7 years ago

These are not yet implemented, go for it!

varung97 commented 7 years ago

Hi guys! I'm new to open source software, but would love to contribute to your project. Is it okay if I try implementing better_and_better, worse_and_worse2 and worse_and_worse3, from the PRISON PROJECT?

drvinceknight commented 7 years ago

Hi @varung97! Welcome to the project!

Those are up for grabs so it would be great to have them in.

If we can help please let us know. We have a gitter room here: https://gitter.im/Axelrod-Python/Axelrod feel free to drop by :)

varung97 commented 7 years ago

Thanks @drvinceknight :D

janga1997 commented 7 years ago

@drvinceknight Apart from type-hinting, I think I would like to work on a strategy. Are these taken or already done?

cautious copycat craby forgetful golden

drvinceknight commented 7 years ago

Hey @janga1997 sorry I saw this when I was busy and then it slipped my mind :)

I've taken a quick look through the matlab files that I seem able to read and to the best of my ability it looks like they're not done yet.

Probably best course of action is that once you identify a particular strategy if you can briefly say what it does we should be able to be a bit more confident in our ability to identify it :)

FYI here's the list of all the strategies currently in the library: http://axelrod.readthedocs.io/en/latest/reference/all_strategies.html We're slowly working towards including alternative names on there too (done for some of them).

Chadys commented 7 years ago

Hello ! I will be working on implementing those strategies :

drvinceknight commented 7 years ago

Welcome @Chadys :)

Take a look at the strategy index that has everything currently implemented here: http://axelrod.readthedocs.io/en/latest/reference/all_strategies.html

the generalization NTitsForMTats

We have a generic LookerUp strategy that could be used to make this :)

S (Win-stay lose-shift, Nerzhin): It cooperates if and only if it and the opponent both played the same strategy on the last round. (from No-tricks)

This is already implemented as WinStayLoseShift. (As a memory one strategy)

memory-based strategies

We have quite a few strategies implemented that make use of memory. We have a generic class for Memory One strategies for example and also much longer member strategies. If you had specific ideas feel free to let us know and we might be able to advise :) :+1:

Chadys commented 7 years ago

Sorry about the WinStayLoseShift, didn't read the index carefully enough I guess. Yes, using LookerUp is an idea, I'll think of a few solutions before deciding how to implement that (I'll probably ask for advice on gitter). Based on http://www.briannakayama.com/Research/UCNCpres.pdf, I thought about doing a generic strategy you give a number to as argument and that create a lookup table (using LookerUp here obviously) by converting it to binary. The length of the memory would then depend on the log of the given number.

marcharper commented 7 years ago

@Chadys Welcome! FWIW I think it's probably better to implement NTitsForMTats directly without Lookerup, since the table could be quite large. I advise making the initial move C in line with the other variants, but you can make it an option if you'd like.

jtsmith2 commented 7 years ago

I can take care of the following from the PRISON list:

Also, I think soft_tf2t is just an instance of a more generic NTitsForMTats that @Chadys is working on where N=1 and M=2. Basically it responds with a single defection only if the opponent has defected on the last two turns and otherwise cooperates.

jtsmith2 commented 7 years ago

Looks like better_and_better, soft_tf2t, and gradual_killer are already done so you can strike those. I'll be better about checking first in the future.

drvinceknight commented 7 years ago

I'll be better about checking first in the future.

Not at all, we try and keep the issue itself up to date :) I've struck those out now.

jtsmith2 commented 7 years ago

I'm going to take a look at the FORTRAN code from the 2nd Axelrod tournament. I spent some time in gradschool translating an old genetic algorithm from FORTRAN into python so hopefully I can be of some use there. I know there was a moment on the Talk Python podcast where you guys said someone had already done some work in this area. Is there any list of which ones have already been implemented?

jtsmith2 commented 7 years ago

I started working my way through the FORTRAN code and I've gotten some (pythonish) psuedocode for the first 6 algorithms pasted in a gist here. I'll likely get a few more on the gist and then start actually writing the code, tests, etc. for those and then mark them as completed and continue on.

drvinceknight commented 7 years ago

That's awesome @jtsmith2!

As well as the PyCon person who worked on the Fortran code @marcharper had a go a little while ago, here's an issue where he described what insights he managed to get #381

GGOUSSEAUD commented 7 years ago

Hello! I'm glad to announce that I'll start working on BM(Bush Mosteller) starting from now on, I think I've gotten the whole theory down and am pretty confident that I can try to implement it. Here's my source: http://www.intechopen.com/books/reinforcement_learning/dynamics_of_the_bush-mosteller_learning_algorithm_in_2x2_games

If any of you has any idea for l (learning rate) value,it is not given so I think I'll start with a solid 0.5 and make some tests. If I need any help I'll pass by gitter or post something here if necessary. Cheers! -Gaëtan

marcharper commented 7 years ago

@jtsmith2 There is a strategy file axelrod_second.py that has the strategies from that tournament, with a few exceptions (e.g. TitForTat is in a different file, it's famous). Note that a few of the strategies also appeared in the first tournament, see axelrod_first.py. IIRC there should be ~40 strategies from the second tournament that are not implemented in the library.

Once upon a time I worked on a few of the strategies from the second tournament, see this quite old branch. I never merged them because I wasn't sure if they were correct or complete. We're going for a "best effort" translation of the original strategies so they don't have to be perfect but good tests will be greatly appreciated.

Let me know if you have any questions!

marcharper commented 7 years ago

@GGOUSSEAUD I recommend experimenting with the learning rate to try to find something good. Just run some tournaments and see what seems to work best.

marcharper commented 7 years ago

@jtsmith2 If it's possible to run the Fortran still we could test each player's response to say the first possible 8 or 16 histories -- that is every permutation of opponent plays C and D of length 3 or 4 -- and use those values as tests.

edouardArgenson commented 7 years ago

Hello, I've read the DBS article and started to work on an implementation

marcharper commented 7 years ago

@edouardArgenson Great! Let us know if you need help writing tests (or anything else).

MariosZoulias commented 7 years ago

Hello .

Can i work on Stein and Rapoport strategy ??? If so , can you give me a guide (i suppose i have to add a new file and one more for testing it) ???

http://axelrod.readthedocs.io/en/latest/reference/overview_of_strategies.html#stein-and-rapoport " Stein and Rapoport Not implemented yet

This strategy plays a modification of Tit For Tat.

It cooperates for the first 4 moves. It defects on the last 2 moves. Every 15 moves it makes use of a chi-squared test to check if the opponent is playing randomly. This strategy came 6th in Axelrod’s original tournament. "

Thank you

drvinceknight commented 7 years ago

Hi @MariosZoulias, this strategy can go in axelrod/strategies/axelrod_first.py: that's where all the strategies for the first tournament go. Similarly the test file for it can go in to axelrod/tests/strategies/test_axelrod_first.py.

Here's the general guidelines on writing a new strategy but let me know if you'd like more guidance: http://axelrod.readthedocs.io/en/latest/tutorials/contributing/strategy/index.html

I think the main "tricky" thing with this strategy will be implementing the chi-squared test which I suggest you do by adding scipy as a dependency https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.chisquare.html

drvinceknight commented 7 years ago

And when you've made the strategy be sure to update the docs at http://axelrod.readthedocs.io/en/latest/reference/overview_of_strategies.html#stein-and-rapoport to say that the strategy is now implemented :D

MariosZoulias commented 7 years ago

@drvinceknight Thank you so much. I will start working as soon as possible