Generate the basic structure of the Lua code in Java

[benelot]

Some of Ben's and Phillip's thoughts on that topic:

We could use a Lua to UML converter to get the basic structure and then generate code from UML to Java.

This will NOT convert the whole code from Lua to Java, but it will give us a stucture we can extend to the finished program. By that we can easily assign methods (functions of classes) to groups that will port that particular method.

Here is the Deepmind Link to access the original source code (in Lua, which is very similar to python) https://sites.google.com/a/deepmind.com/dqn/

I forked (copied in a git way) a repository from somebody that already looked at the code, made some minor tweaks and could reproduce Deepmind's results: https://github.com/benelot/DeepMind-Atari-Deep-Q-Learner

A short research on the Lua-2-UML-2-Java conversion:

Plantuml can build UML diagrams from Lua, we might be able to generate Java from that: http://plantuml.com/

Otherwise we might use this: http://fperrad.github.io/lua-Coat/uml.html

To generate Java code from UML, we could use Visual Paradigm, which unfortunately is a commercial software, but there is a community version for non-commercial use: http://www.visual-paradigm.com/download/community.jsp https://www.visual-paradigm.com/tutorials/sdeeccodegen.jsp

Otherwise we can probably use this here: http://marketplace.obeonetwork.com/module/uml2java-generator https://github.com/ObeoNetwork/UML-Java-Generation

[dannyneil]

Hey guys! Definitely looks cool, but be careful about being too rigid - we can't do a straight code port. I'm still mulling over it, but I know the following things will be issues:

• With I/O specifically, we will have a very different interface -- DeepMind exclusively uses the standard video output, while there is no visual environment for GVGAI by default (though we can of course render one).
• We won't be using ALE, so our callbacks and control methods are significantly different.
• We don't have access to the torch tensor library, so we will need to find a decent replacement for Java (I've used JBLAS in the past to help, but we'll probably need to find something with better convolution support), so we will need to rewrite the matrix routines.

Hopefully, within the actual learning environment it won't be too different -- and thankfully, the agent isn't actually very complicated. Regular Q-learning with a playback memory in it and a really smart function approximator (the convnet). Just things to keep in mind!

[pkainz]

Results on the lua-Coat approach

Unfortunately lua-Coat cannot generate UML from a simple Lua script directly. Classes must be declared in a certain way using the Lua 'Coat' module, see example. Furthermore, in the available deepmind code, they do not declare 'high-level' Lua classes themselves, but use or derive classes from the Torch ML library. Either way, easy abstraction without manually adding annotations on the classes' inter-dependencies will not work.

My guess is, that this will be the same problem for the PlantUML approach, but I'll look into that and report it later here.

Cheers, Phil

[pkainz]

Results on PlantUML

Same issue as expected. All inter-class dependencies have to be declared in a special way, i.e. in lua-style comments within the class:

--[[
  @startuml
  Class A <|-- Class B
  @enduml
]]

As a consequence, and since we want to generate these dependencies from the existing deepmind code, we would have to manually re-annotate all files. A lot of the code depends on Torch and I don't expect modeling these functions to be straightforward. Moreover, I think it will take more time to have a clean UML representation than to actually somehow on-the-fly-port this implementation to Java.

To conclude, PlantUML doesn't help simplifying the port to Java either, so it seems that we will have to work our way through the DQN architecture and abstract it manually, so that we can figure out how parts are working together.

[benelot]

Ok, thanks for the update. Sad to hear that it is harder to do than we thought. Let us look into the code before the next meeting. Furthermore I believe that we need to look for Torch alternatives to replace the missing torch components. During my own research I found deeplearning4j (http://deeplearning4j.org/), where I saw a tensor network implementation, but I have no idea whether this could be a replacement and what other components we need.

[benelot]

I think these are interesting pre-hackathon insights, so I think we can close this issue because we found out that the approach is the wrong one. We can not port one-to-one.