search

schrum2 / MM-NEATv2

MM-NEAT version 2.0 is no longer supported. Please get MM-NEAT 3+ from https://github.com/schrum2/MM-NEAT
Other
11 stars 5 forks source link

Atari 2600 emulator #290

Open schrum2 opened 7 years ago

schrum2 commented 7 years ago

Lately, there has been an explosion of interest in using Deep Reinforcement Learning to learn to play Atari 2600 games. I would like to get in on this trend by integrating this code into MM-NEAT. MM-NEAT is more focused on evolution, but there are opportunities to combine and compare it with Deep RL as well.

Annoyingly, the Arcade Learning Environment, which is a system for programming agents for Atari games, is C++ based. However, there is a Java interface to the compiled program.

The original ALE page is here: http://www.arcadelearningenvironment.org/

However, the source code is in C++ and they only really provide guidance for compiling it on Mac and UNIX.

This webpage, https://github.com/Islandman93/Arcade-Learning-Environment, has the ALE C++ code in a Visual Studio project (Visual Studio is a Microsoft Windows IDE for C++ and more). We could download this, compile it to an executable, and then interface with the resulting executable via Java.

This page also has a tutorial on how to compile the original ALE code in Windows, but it looks complicated: https://medium.com/@AtariAnn/running-a-l-e-arcade-learning-environment-0-4-4-on-windows-49f2494b5344

And another tutorial for compiling using Visual Studio: https://groups.google.com/forum/#!topic/arcade-learning-environment/WMCrtTZPE2A

There are other approaches though, like this: https://deeplearning4j.org/reinforcementlearning This tool claims to be a Java interface for the Open AI Gym: https://gym.openai.com/

We could also skip the ALE and simply try an emulator for Atari. The C++ emulator used by ALE is stella, but there is a Java version called JStella: http://jstella.sourceforge.net/ I guess it might be easier to start with an emulator and figure out how to make agents than to start with a C++ program and figure out how to interface with it. It's hard to tell.

Javatari is another Java Atari emulator: https://github.com/ppeccin/javatari

There are many possibilities!

schrum2 commented 7 years ago

This issue probably requires a lot of looking around and considering possibilities before we actually put any code in the project. If anything, you might want to start with the deeplearning4j link and start working through some tutorials (might even want to use IntelliJ instead of Eclipse, as suggested).

This is a low priority issue though

schrum2 commented 7 years ago

No one is directly assigned to this any more. It's something that I hope someone will get around to, but will be tough compared to lots of others things we could do instead.