OpenAI Retro Contest

[joyhuang9473]

OpenAI Retro Contest

• Author: openai
• Origin: https://contest.openai.com/details
• Related:
  • Retro Contest: Results https://blog.openai.com/first-retro-contest-retrospective/
  • Gotta Learn Fast: A New Benchmark for Generalization in RL https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/retro-contest/gotta_learn_fast_report.pdf
  • https://blog.openai.com/retro-contest/
  • https://github.com/openai/retro-contest
  • https://github.com/openai/retro-movies
  • https://github.com/openai/retro-baselines
  • OpenAI Retro Contest – Compilation of Reinforcement Learning Write-Ups http://www.noob-programmer.com/openai-retro-contest/reinforcement-learning-write-ups/
  • 1st place: OpenAI Retro Contest – Guide on “How to Score 6k points on Leaderboard in AI Challange”
  • http://www.noob-programmer.com/openai-retro-contest/how-to-score-6k-in-leaderboard/
  • https://github.com/eyounx/RetroCodes
  • 2nd place: https://github.com/xupe/mistake-in-retro-contest-of-OpenAI
  • 3rd place: https://github.com/aborghi/retro_contest_agent
  • 8th place: https://github.com/fgvbrt/retro_contest
  • 15th place: https://medium.com/@olegmrk/openai-retro-contest-report-b870bfd014e0
  • OpenAI Retro Contest – Everything I know about JERK agent http://www.noob-programmer.com/openai-retro-contest/jerk-agent-algorithm/

[joyhuang9473]

[OpenAI Retro Contest] Getting Started https://medium.com/@deankayton/openai-retro-contest-getting-started-62a9e5cc3801

[joyhuang9473]

Day one of the OpenAI Retro Contest. https://medium.com/@tristansokol/day-one-of-the-openai-retro-contest-1651ddcd6aa5

[joyhuang9473]

(reward function) More information in the rewards section of this:

https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/retro-contest/gotta_learn_fast_report.pdf

[joyhuang9473]

OpenAI Retro Contest – Everything I know about JERK agent http://www.noob-programmer.com/openai-retro-contest/jerk-agent-algorithm/

[joyhuang9473]

SonLVL (Pallets, Tiles, Blocks, Chunks, Solidity Maps, Foreground, Background)

• http://info.sonicretro.org/SonLVL
• https://github.com/sonicretro/SonLVL

[joyhuang9473]

@PixelyIon

The stuff of Blocks, Solidity Maps, Tiles: https://cdn.discordapp.com/attachments/433354549835595776/444272473702137856/Sanic.zip

[joyhuang9473]

@Garreth

What I understand is (Tell me if any of the following is wrong):
- Reward function cannot be changed
- The main purpose of the contest is to find a way that generalize well and fast (that's why we don't have access to info dictionary in test)
- JERK is simple but achieve better result at early stage (because it is guided)
- POO and Rainbow are slower, but should eventually get better result than JERK if we train more
- The main problem is generalization

What I think we should do is finding more useful features like @◱ PixelyIon is trying to do.

[joyhuang9473]

train set: https://contest.openai.com/static/sonic-train.csv test set: SpringYardZone.Act1

[joyhuang9473]

@sl

ConvVAE for RL

score: 5302.79

example: https://keon.io/deep-q-learning/

[joyhuang9473]

@unixpickle

@m1234d as it is, Rainbow performs better than PPO. I don't think either algorithm is the best for this task, especially since they are very much worse than humans (so humans represent at least one better algorithm)

[joyhuang9473]

@lyons Wrote a quick post about using retrowrapper to run multiple environments at once

https://mikelyons.org/2018/05/22/Multiple-Retro-Environments.html

[joyhuang9473]

@lyons running retro in colab https://drive.google.com/file/d/11Mxg30mXEvhk8jB0iJ-cFw1k0wICkf8e/view?usp=sharing

[joyhuang9473]

@Rezix @sulo If it helps i made a big write-up based on Jek-Agent: http://www.noob-programmer.com/openai-retro-contest/jerk-agent-algorithm/

[joyhuang9473]

@tristansokol OpenAI Retro Contest tools! https://gist.github.com/tristansokol/062b1d509e2e8e6e250a30ae09928a58

[joyhuang9473]

@Sugaku Retro Games in Gym: https://github.com/rfurman/retro

[joyhuang9473]

@Daniel

[writeups] Using Deep Reinforcement Learning to Play Sonic the Hedgehog: An attempt to replicate the World Models paper to play Sonic for the OpenAI Retro Contest.

https://medium.com/@mrdbourke/the-world-model-of-a-hedgehog-6ff056a6dc7f

[joyhuang9473]

@flyyufelix

[writeups] Train a Reinforcement Learning agent to play custom levels of Sonic the Hedgehog with Transfer Learning https://flyyufelix.github.io/2018/06/11/sonic-rl.html

[joyhuang9473]

@Dylan

[writeups] World Models applied to Sonic

• https://dylandjian.github.io/world-models/
• https://github.com/dylandjian/retro-contest-sonic

[joyhuang9473]

OpenAI Retro Contest – Compilation of Reinforcement Learning Write-Ups http://www.noob-programmer.com/openai-retro-contest/reinforcement-learning-write-ups/

[joyhuang9473]

@seungjaeryanlee

https://github.com/seungjaeryanlee/retro-agents

[joyhuang9473]

Best Write-ups

RANK	WINNER	WRITEUP
#1	Dylan Djian	World Models
#2	Oleg Mürk	Exploration algorithms, policy distillation and fine-tuning
#3	Felix Yu	Fine-tuning on per-zone expert policies