Error running supplied code for random policy

[adamjosephjensen]

After following the setup instructions, when I try to run:

from osim.env import RunEnv

env = RunEnv(visualize=True)
observation = env.reset(difficulty = 0)
for i in range(200):
    observation, reward, done, info = env.step(env.action_space.sample())

Then I get:

Traceback (most recent call last):
  File "test.py", line 4, in <module>
    observation = env.reset(difficulty = 0)
  File "/Users/adamjensen/anaconda2/envs/opensim-rl/lib/python2.7/site-packages/osim/env/run.py", line 68, in reset
    super(RunEnv, self).reset()
  File "/Users/adamjensen/anaconda2/envs/opensim-rl/lib/python2.7/site-packages/gym/core.py", line 70, in reset
    raise NotImplementedError
NotImplementedError

simbody-visualizer: received Shutdown message. Goodbye.

It looks like the osim environment class is calling super(RunEnv, self).reset(), which is never meant to be called because the Gym class is never meant to be called directly.

I'm interested in using this package to investigate reinforcement learning. I'm trying to use it for a class project in CS234 at Stanford. I know the competition is closed, but I had hoped that the code would still be available to play around with. Is that not the case?

[Frawak]

It's Gym that causes it. At the end of January, they had a clean-up with this commit: https://github.com/openai/gym/commit/4c460ba6c8959dd8e0a03b13a1ca817da6d4074f#diff-527c178de63067298819918734912f90

As long as osim-rl has not adapted their code I would recommend rolling back to the commit prior to the one linked above. Or you remove the underscore of the reset method in OsimEnv.

There, the reset was called as follows:

Call of reset in RunEnv:

def reset(self, difficulty=2, seed=None):
        super(RunEnv, self).reset()
        ...

The super in gym.Env:

def _reset(self): raise NotImplementedError
...
def reset(self):
        return self._reset()

And the overwrite of _reset in OsimEnv:

def _reset(self):
        self.istep = 0
        self.osim_model.initializeState()
        return self.get_observation()

Now, as can be seen in the Gym commit, the underscore method, which was overwritten by OsimEnv, is removed.

[adamjosephjensen]

@Frawak thanks for your comment. I tried removing the underscore from OsimEnv here: https://github.com/toothlessdragon/osim-rl/commit/18c219d0a55b13306869eafd2d62d4c4933ba25f

But for some reason it still failed to run.

[Frawak]

You're probably not using the OsimEnv in your repository. If you followed the instructions here, you would have set up conda with a new environment (here: opensim-rl). You are importing mainly from the packages installed there. E.g.: when you're writing import osim in a script using that environment, it links to [some directory]\Anaconda2\envs\opensim-rl\lib\site-packages\osim and not to what you have in your repository. You would have to manually adjust the import.

But before you change source code or some other complicated solution, I still recommend changing the version of the gym package.

1) Bring up your command prompt and activate the conda environment: source activate opensim-rl 2) conda list brings up the list of packages in that environment. gym may be listed with version 0.9.6 which is the clean-up commit. 3) To change the version just type pip install gym==0.9.5 That should uninstall the current version as well.

I hope this is doing the trick.

[spMohanty]

Hi @toothlessdragon,

As @Frawak pointed out, these issues are indeed because of the breaking changes from Gym earlier this month.

An easy fix for now is to ensure that you have Gym version v0.9.5 installed when trying to install osim-rl.

I just pushed a few changes to the repository to ensure that on every installation you only reference the correct version of Gym. This is a temporary fix.

A more permanent fix requires some more internal refactoring, which we will include in the next release, along with more models+tasks.

@kidzik : Keeping this issue open, as this should ideally be closed after the proper refactoring to deal with the breaking changes in gym > 0.9.5

[kidzik]

Fixed for the new 'generic' environment https://github.com/stanfordnmbl/osim-rl/blob/2c3d46ab108547d8192b6acd25b8332d79fe3a92/osim/env/generic.py