search

carykh / PrisonersDilemmaTournament

Watch This Place's awesome video about iterated Prisoner's Dilemma for context! https://www.youtube.com/watch?v=BOvAbjfJ0x0
MIT License
205 stars 159 forks source link

Update from Cary! (I implemented subprocesses) #89

Open carykh opened 3 years ago

carykh commented 3 years ago

Hi everyone! I'll post Prisoner's Dilemma Tournament updates here from now on, because this place is more specific to this project than the carykh Community posts.

Anyway, I've finished writing the code that launches 10 subprocesses that each run the 1500x1500 strategy matrix. This means I should be able to get 10 runs done each night. (I have 10 runs done so far, and the goal is to finish 100, and then average the results.) Most of you said you liked the idea of a quick-n-easy results reveal on a secondary channel (probably lazykh: https://www.youtube.com/channel/UCobNHsxWdWDFCuPqHoBmyUw), so as soon as the 100 runs are complete, I'll probably make that video within a day or two!

I was considering voice-calling people who submitted any strategy that stood out before publicly revealing results, and then getting their "surprised reaction" when they discovered they'd won. But now, I don't think I'll do that. My improved plan is to:

Besides, carykh isn't a reaction channel, it's a interesting-CS-projects channel! Cool cool

arielsysdev commented 3 years ago

Sounds like a plan! I can only imagine trying to understand thousands of thought processes in the thousands of strats. Even if you don't think you should publish all of the strats, can you at least publish the raw data of the results? I want to look around for interesting patterns as well, maybe it could also help your research!

carykh commented 3 years ago

can you at least publish the raw data of the results? I want to look around for interesting patterns as well, maybe it could also help your research!

Yes, I do want to do that! I'll probably post the 1500x1500 matrix of average scores of each strat vs every other strat, and then I'll also include the more innocuous data about each strategy, such as the code's runtime, length in lines of code, time it was submitted, and birth year of the respondent (if they were cool with sharing it.) There might be correlations people can find in the data long after I'm done with it!

arielsysdev commented 3 years ago

Thanks! I'm also thinking if you could extract at least the comments in the file? That could give a good idea of what people had in mind even if we can't see the codes themselves. I don't think people would take issue with that since their IP would still safe. Up to you though.

carykh commented 3 years ago

I'm also thinking if you could extract at least the comments in the file?

I could perhaps do that, although I can't guarantee it! With 1,600 entries, whatever method I do decide to use to extract the comments of a file would have to be automated. Perhaps I could display the first lines of the code that are 100% comments (with a # or ''' at the start), but then that wouldn't include everyone who put comments only after the imports. There may be a more foolproof way to only collect comments that I haven't discovered yet, (if there is, let me know where to find it!) But either way, it will be an involved process. I'll see!

karapapas commented 3 years ago

I'm also thinking if you could extract at least the comments in the file?

I could perhaps do that, although I can't guarantee it! With 1,600 entries, whatever method I do decide to use to extract the comments of a file would have to be automated. Perhaps I could display the first lines of the code that are 100% comments (with a # or ''' at the start), but then that wouldn't include everyone who put comments only after the imports. There may be a more foolproof way to only collect comments that I haven't discovered yet, (if there is, let me know where to find it!) But either way, it will be an involved process. I'll see!

There is this library https://pypi.org/project/comment-parser/ but I guess it probably looks for a specific patterns, for example comments included in triple-quotes at the beginning of methods, etc.

Also, in my opinion the problem with the code disclosure might also apply to comments, honestly I do not know, but if you'd ask me I'd be more embarrassed to show my comments than my code. :D

ThatXliner commented 3 years ago

There's also Python's tokenize module

carykh commented 3 years ago

There is this library https://pypi.org/project/comment-parser/ but I guess it probably looks for a specific patterns, for example comments included in triple-quotes at the beginning of methods, etc.

Ooh, that looks pretty helpful, thanks for pointing it out karapapas!

Although, part of me is wondering if it is okay to just release the code verbatim. I mean, that would be simpler, and the applicants did submit their code to a public competition with public results!

Actually, here's a thought... on the submission form, I asked "If you do win something, is it cool if you and I talk really quick over voice-chat, for a video?"...and ~80% of people submitted "Yes". Maybe it's safe to assume that those 80% are cool with their code appearing in the vid, too? That would give curious spectators a decent sampling of the strategy pool! But, I guess someone could be cool with a video, but not cool with their code being shared, lol