Documentation?

[yxie20]

Thank you for releasing this work. I sat down with the code for a few days now and I'm having a hard time figuring out how to correctly change the game parameters. Is there documentation available for folks who didn't develop this? Excited to incorporate Toybox in our own research.

[jjfiv]

• There's a minimal interface shown in this colab notebook: https://colab.research.google.com/drive/1MMPseXpJ5esIXwTcHo0EBSG2iLaR0JPB
• Tests are also fairly decent: https://github.com/toybox-rs/Toybox/blob/master/test/benchmark.py
• Slightly out of date fancy-docs: http://toybox.rs/docs/ctoybox/

[yxie20]

Thank you. However, these examples do not show how one can access toybox through OpenAI gym, while also changing game parameters (i.e. satisfy R3 and R4 "software requirements" simultaneously).

[jjfiv]

We have gym environments, e.g., https://github.com/toybox-rs/Toybox/issues/157

But we had to edit the baselines repo to get to use it's atari wrappers (FireResetEnv, etc.). We've been intending to move off, to e.g., stable_baselines, but we haven't. We have heard it works (after merging https://github.com/toybox-rs/Toybox/pull/176)

In my experience, most RL libraries never expect another env LIKE atari, so it involves digging up a fair bit of private classes to get those wrappers. I think openai/gym has them public, now, but 🤷

Depending on what library you're using, figuring out the many wrapping layers (especially if you want to use pixels!) ... that's basically all the real work.

The colab shows how to get both features and images.

[drmeerkat]

How can I modify the game environment as described in the paper? And what are the available modification choices? Thank you!

[jjfiv]

For that a good starting point would be to dump the state json:

with Toybox("breakout") as tb:
  print(tb.state_to_json())

The types.rs for the particular game functions as a schema: (We also have JSONSchema around, but I don't know folks who are comfortable reading that, tbh).

https://github.com/toybox-rs/toybox-rs/blob/master/tb_breakout/src/types.rs#L84-L111

Anything in that state can be changed and immediately reloaded into the system.

[yxie20]

Thank you. But with breakout loaded as a gym environment, how can I access the state_to_json() function and change the game as you suggested? Or are you suggesting that satisfying "software requirements" R3 and R4 simultaneously is not supported out of the box with this work?

[jjfiv]

It totally is. Pro-tip, look around your objects.

There's a .toybox or ._toybox on the gym env, try using dir(env), but be careful about wrappers.

[jjfiv]

https://github.com/toybox-rs/Toybox/blob/08ff3bc11866696adbba72d556646b3641fef17e/toybox/envs/atari/base.py#L38-L48

[yxie20]

Got it! Thank you for the pointers.

I'll include a minimal example below for others who may find it useful:

import torch
from toybox import Toybox, Input    # @yxie20
import gym
from matplotlib.pyplot import imshow
get_ipython().run_line_magic('matplotlib', 'inline')

env_id = "BreakoutToyboxNoFrameskip-v4"
env = gym.make(env_id)
assert 'NoFrameskip' in env.spec.id
# dir(env.toybox)

state = env.toybox.state_to_json()
imshow(env.toybox.get_rgb_frame())

for brick in state['bricks']:
    brick['position']['y'] += 40

env.toybox.write_state_json(state)
imshow(env.toybox.get_rgb_frame())