Wrapping universe

[justheuristic]

By default Universe provides a low-level keyboard control with very high-dimensional action space AND a high-dimensional image + occasional other stuff as observations.

While it is a noble quest to tackle the environment as it is, we better first try learning something in a more RL-friendly setup:

• smaller-resolution image
• discrete action space: wrap only the essential actions like "turn car left/right", not "move cursor to that random location"
• repeat the same action for N (e.g.4) timesteps to make effective sessions shorter

So, we need to create a wrapper environment that takes openai universe game and alters it's reset() and step() methods to make env easier for agent to master.

Here's one game that seems simple enough for this, i suggest sticking to it unless we bump into some unsurpassable obstacle.

One example of altering an environment this way can be found here

[justheuristic]

This is probably achievable through some of the wrappers https://github.com/openai/universe-starter-agent/blob/master/envs.py#L9

More specifically, something like this https://github.com/openai/universe-starter-agent/blob/master/envs.py#L211

[justheuristic]

Made most basic wrapper that just spawns one container per player: https://github.com/yandexdataschool/tinyverse/issues/13 It's probably possible to play much faster with some universe tricks - that's now a priority.