Get a Box2D domain working

[schrum2]

The Box2D domains in OpenAI Gym are also pretty interesting.

https://towardsdatascience.com/how-to-install-openai-gym-in-a-windows-environment-338969e24d30

[nazaruka]

This solution's packages installed smoothly, all except for mujoco.py and atari-py. I will be seeing how well Box2D works after lunch

[nazaruka]

I found an interesting connection between this issue and issue 3, which has to do with trying to implement various classic_control domains into DQN_cartpole.py. It turns out that the pendulum.py environment has a completely different setup for its action space than the other classical_control domains: while the others set them up using an array or with the aid of gym.spaces.discrete(), pendulum.py and the Box2D domains use the Box object.

As mentioned in my comment on issue 3, I corrected env.action_space.n to env.action_space.shape[0] and got it to render, but it crashed immediately with the following error:

Traceback (most recent call last):
  File "DQN_cartpole.py", line 152, in <module>
    obs1, rwd, done, _ = env.step(act) # execute this action and observe the resulting reward and image
  File "c:\users\admin\desktop\southwestern\scope\files\repo\gym\gym\envs\classic_control\pendulum.py", line 37, in step
    u = np.clip(u, -self.max_torque, self.max_torque)[0]
IndexError: invalid index to scalar variable.

There are some oddly similar errors I find when running this corrected code with Box2D domains:

• BipedalWalker-v2 has an issue at line 152 as well, but it ends with:

  File "c:\users\admin\desktop\southwestern\scope\files\repo\gym\gym\envs\box2d\bipedal_walker.py", line 385, in step
  self.joints[0].motorSpeed     = float(SPEED_HIP     * np.sign(action[0]))
  IndexError: invalid index to scalar variable.

  While it is true that CarRacing-v0 and LunarLander-v2 have their own issues, it seems as though fixing this one could solidify the link between Pendulum-v0 and the Box2D domains, thus knocking two birds out with one stone.

[schrum2]

Can we get a solid list of the Box2D domains that you have been able to successfully run in this issue thread? I just want an update on where we are at. For any ones that work, making them accessible via the central execution point #8 is a fairly high priority.

[schrum2]

I know some of these work for you. Please list all domains that work in this issue thread. You might want to incorporate code from the Open AI baselines instead of the tidy RL code we were using.

[schrum2]

Please provide a list of working domains so you can close this issue.

[nazaruka]

The ones that work are BipedalWalker-v2 and LunarLander-v2. The others load the environments but crash immediately; this behavior may have to do with certain algorithms' implementations of different types of action spaces (Discrete vs. continuous). The crashing issues seem to relate primarily to how algorithms attempt to manipulate said action spaces; if the environment declares some of incorrect size, the algorithm will not execute.