carykh / PrisonersDilemmaTournament

Watch This Place's awesome video about iterated Prisoner's Dilemma for context! https://www.youtube.com/watch?v=BOvAbjfJ0x0
MIT License
205 stars 160 forks source link

Discussion: What do you all predict will be the most common tactics? #26

Open jherndon8 opened 3 years ago

jherndon8 commented 3 years ago

I personally predict lots of submissions will be variants on tit-for-tat. Commenters in youtube speculate lots of people will submit the random one-liner.

StampDanFan commented 3 years ago

Same, I also think that many submissions will be variants on tit-for-tat. However, as Carykh has many Machine Learning videos, I think that some submissions might use Machine Learning. I don't think the random one-line submission will win, and hopefully it doesn't.

It will be interesting to see how the tournament turns out!

jherndon8 commented 3 years ago

No way a random is gonna win. If there’s several tit-for-tatters out there, they’ll all defect half the time with the randos, drastically reducing the rando’s score.

hand-burger commented 3 years ago

Grim Trigger, being the best strategy, I think will be the most iterated upon, but maybe not the most common. So yea, probably tft.

willow-iam commented 3 years ago

Grim Trigger, being the best strategy, I think will be the most iterated upon, but maybe not the most common. So yea, probably tft.

What makes you think it's the best strategy?

hand-burger commented 3 years ago

Grim Trigger, being the best strategy, I think will be the most iterated upon, but maybe not the most common. So yea, probably tft.

What makes you think it's the best strategy?

it consistently gets first place. that's why.

WalterCM commented 3 years ago

Grim Trigger, being the best strategy, I think will be the most iterated upon, but maybe not the most common. So yea, probably tft.

What makes you think it's the best strategy?

it consistently gets first place. that's why.

tisfortat actually consistently gets first place. Doesn't grim trigger lose too many points against randoms and detective strategies?

willow-iam commented 3 years ago

Grim Trigger, being the best strategy, I think will be the most iterated upon, but maybe not the most common. So yea, probably tft.

What makes you think it's the best strategy?

it consistently gets first place. that's why.

tisfortat actually consistently gets first place. Doesn't grim trigger lose too many points against randoms and detective strategies?

Grimtrigger beats random, but it loses points from titfortat.

hand-burger commented 3 years ago

Grim Trigger, being the best strategy, I think will be the most iterated upon, but maybe not the most common. So yea, probably tft.

What makes you think it's the best strategy?

it consistently gets first place. that's why.

tisfortat actually consistently gets first place. Doesn't grim trigger lose too many points against randoms and detective strategies?

Every test I run, grim tripper always gets first, not sure how its different for you.

StampDanFan commented 3 years ago

Every test I run, grim tripper always gets first, not sure how its different for you.

I don't know why that would be, tit-for-tat always wins when I run the simulation. It says on the htwins website that tit-for-tat usually wins: image

hand-burger commented 3 years ago

Every test I run, grim tripper always gets first, not sure how its different for you.

I don't know why that would be, tit-for-tat always wins when I run the simulation. It says on the htwins website that tit-for-tat usually wins: image

I figured out why my results were different, I had an older file which I had made before which was affecting the results, so that really shows how the actual run is going to be very different than any practice runs.

jherndon8 commented 3 years ago

I figured out why my results were different, I had an older file which I had made before which was affecting the results, so that really shows how the actual run is going to be very different than any practice runs.

Good lord, what other strategies did you put in that put grimTrigger in first?!?

hand-burger commented 3 years ago

I figured out why my results were different, I had an older file which I had made before which was affecting the results, so that really shows how the actual run is going to be very different than any practice runs.

Good lord, what other strategies did you put in that put grimTrigger in first?!?

So it was called "beNice" and on the first turn it defects, then checks what the opponent did on that turn, if they were nice (cooperated), I would cooperate the whole time, if they were mean (defected) I would be mean for every remaining turn. I completely forgot about it and it just really changed the leaderboard.

l4vr0v commented 3 years ago

As I outlined in my comment on #27, I think the optimal solution here is a "forgiving" tit-for-tat (not ftft)- one that doesn't defect unprovoked, responds to unprovoked defections with a defection on the next turn (or, in light of beNice, two turns later), and at some point hands out olive branches to its opponent to see if they'll go back to cooperating (but doesn't do this too aggressively).

That last piece- the forgiveness mechanic- is the unsolved component of this problem. I expect people who will submit things to do so with the intention of winning, so I suspect the meta will be dominated by tit-for-tat variants and specifically tit-for-tat variants with forgiveness rules. This will lead to boring 3/3 matches in most cases but the forgiving tit-for-tats will compete against one another based on how well they handle the strategies that defect unprovoked but have some deterministic component that can be used to largely maintain a C/C loop and/or recover some of the loss by getting a D/C +5 right after the surprise C/D (so detective variants, hand-burger's beNice above, joss-like approaches).

beNice-like strategies might actually play the role of kingmaker since they would heavily advantage delayed titForTat strategies (ones that respond to a C/D with a D/C two or more turns later rather than one) enough to vastly outweigh the small benefit (+2 points each) those titForTat strategies would gain from each detective. Explains why grimTrigger did so well in hand-burger's meta. If beNice is in the mix, it would more or less kill my own submission too. Although if a tit-for-tat variant waits too long before respond to a defection, it might set off a detective-like strategy to defect for the remainder of the game... So tit-for-tat in general can realistically do nothing (which makes it alwaysCooperate), defect the next turn (which beNice would really destroy), or defect in two turns (which would play well with beNice and detective). Or it could have some brittle edge-case logic to accommodate this double-bind that leaves it vulnerable to quirks of the meta.

Someone should really submit beNice to mess with the world and force tit-for-tat variants to have some deferred-response logic. Even a single beNice can outweigh the benefits of a non-deferred tit-for-tat. Then again, someone else could just submit a beNice (or anti-detective) that checks for defections in the first 4 turns and maximally punishes that with defections for the rest of the game, cancelling out the meta-breaking impact of the original beNice.

jherndon8 commented 3 years ago

Someone should really submit beNice to mess with the world and force tit-for-tat variants to have some deferred-response logic. Even a single beNice can outweigh the benefits of a non-deferred tit-for-tat. Then again, someone else could just submit a beNice (or anti-detective) that checks for defections in the first 4 turns and maximally punishes that with defections for the rest of the game, cancelling out the meta-breaking impact of the original beNice.

Oh good god, a couple anti-detectives that are out there just out of spite for the tit-for-tats could really break this game lol

l4vr0v commented 3 years ago

I might be wrong about the non-viability of unprovoked defections. It's basically a wash between grimTrigger and alwaysCooperate (+1 net gain) that's offset by losses when playing most tit-for-tats (minimum 1 point loss, average case much worse). But if you throw in ftft detection and spam DCDCDCDC once you detect an ftft, you may be able to make up for that loss (you basically average 3.9-4 points/move in your ftft match).

I wrote a simple ultimateDetective that does this and it fares pretty well in my tit-for-tat-heavy meta. I'm sure it can be optimized further but perhaps detectives/unprovoked defections aren't the dead end I think they are. If you're able to sufficiently exploit "clever" strategies and trick them into cooperating while you periodically defect, you can get a huge advantage.

hand-burger commented 3 years ago

Someone should really submit beNice to mess with the world and force tit-for-tat variants to have some deferred-response logic. Even a single beNice can outweigh the benefits of a non-deferred tit-for-tat. Then again, someone else could just submit a beNice (or anti-detective) that checks for defections in the first 4 turns and maximally punishes that with defections for the rest of the game, cancelling out the meta-breaking impact of the original beNice.

I did submit beNice, so we'll see how that plays out. Don't tell cary but I submitted two strategies because I realized how bad beNice will be. It'll be interesting to see how it does.

l4vr0v commented 3 years ago

Hahaha well if you submitted beNice I know I have no chance of winning. I submitted npnntt, which probably isn't even my best strategy. But impossible to change now.

In light of that, I think the optimal strategy is a:

Although I'm having trouble thinking of abusive counterplay against, e.g., a deferred tit-for-tat. Anything that can possibly tolerate a tat without responding with a tit is exploitable (e.g., detective during moves 0, 1, ftft) but anything that's guaranteed to respond to each tat with a tit won't let you do better than 3pts/move. Plus there'll probably be punitive strategies beyond grimTrigger that respond to tats with multiple tits, and those could really set you back if there's enough of them in the meta.

hand-burger commented 3 years ago

So I just rewrote beNice to actually do what I wanted it to do and now Simpleton consistently gets first, odd.

l4vr0v commented 3 years ago

@hand-burger You can use the head-to-head file in #31 to figure out why (by looking at which strategies simpleton scores relatively high points against, and what's dragging down the other candidates). I'd do it myself but it looks like your work is not in a public repo.

hand-burger commented 3 years ago

@hand-burger You can use the head-to-head file in #31 to figure out why (by looking at which strategies simpleton scores relatively high points against, and what's dragging down the other candidates). I'd do it myself but it looks like your work is not in a public repo.

I'm not exactly sure what you mean, so I made a public repo with beNice.

edit: Never mind I got it, anyways there's still a repo with beNice.

astropingo commented 3 years ago

My strategy is called grandpa_checking_you-_play. It always tells the truth, and adds "are you winning, kid?" on memory. At the end, if it loses it returns a smile.

I believe this will be the way to go.

nobody5050 commented 3 years ago

Very good strategy! I think it will win every time

Lasermancer commented 3 years ago

I think some people will troll and do a tactic which detects TFT and then always defects, or random which will detect if you defect it too much and then will grudge. Where it comes to best strategy, assuming no trolls, the best is a detective. While a good variant of TFT can handle detective, Joss and beat randomness, detectives can just exploit too forgiving TFTs.

redtachyon2098 commented 3 years ago

I'm using a machine learning based strategy, and I'm not expecting it to win at all, since a well-crafted detective will almost always be better at dealing with the more common strategies. My strategy wins almost every round, but in the process loses too many points, and can't get a top spot at all. Currently I downloaded I4vr0v's fork and I'm training it on some of them, such as all the detective and tit for tat variants, conMan, etc. But mine is just a gimmick, and wasn't expected to win in the first place. After all, I'm participating in a carykh coding challenge. I think my decision to make something machine-learn-y kinda proves that I'm a fan. (btw I'm not good at coding at all, in any way, so that's probably another reason my strategy is so bad.)

SeverinPaar commented 3 years ago

How did you manage to save your model? I am building a machine-learning strategy as well, but I'm struggling with keeping all the data in one python file

redtachyon2098 commented 3 years ago

I took advantage of "memory" ad stored the model there. I made the model a single object using a class, and I just said return {the choice} [{The model}, {other information}]. After that, earlier in the script, i detect if the variable "memory" is a list or not, and if it is, the model = memory[0], and if it isn't, I generate a new one. Of course this isn't the only way to do this, there probably exists a better way to do it.