search

ctmakro / stanford-osrl

NIPS2017 challenge
47 stars 12 forks source link

readme

My codebase for Stanford/EPFL NIPS2017 RL Competition w/ OpenSim

you know, to walk the skeleton.

original DDPG/PPO code from https://github.com/ctmakro/gymnastics

more details at https://github.com/stanfordnmbl/osim-rl

What on earth is EPFL

École polytechnique fédérale de Lausanne (School Polytechnical Federal of Lausanne), the MIT of Switzerland. They seem to speak French.

Dependencies

To Run

$ ipython -i ddpg2.py

then enter r(100) to train the agent for 100 episodes.

For PPO implementation, just change the filename:

$ ipython -i ppo3.py

Parallelism / Farming

Current version requires farming (which is VERY IMPORTANT! Please read https://github.com/stanfordnmbl/osim-rl/issues/58). Before starting ddpg2.py, you should first start one or more farm, preferably with Python 2.7, by running python farm.py on each SLAVE machine you own. Then create a farmlist.py in the working directory (on the HOST machine) with the following content:

farmlist_base = [('127.0.0.1', 4),('192.168.1.33',8)]
# a farm of 4 cores is available on localhost, while a farm of 8 available on another machine.
# expand the list if you have more machines.
# this file will be consumed by the host to find the slaves.

Then start ipython -i ddpg2.py. The farmer should be able to reach the farm on your local machine. You can now type r(100) to train.

if you want to use the farming code for other algorithms/purposes, please see farming_demo.py for an example.

Note for users on Win7 x64 + Python 3.5 (2017-07-06)

Assume you want to run osim-rl on Windows w/py35, since TensorFlow support only Python 3.5 on Windows.

Edit: this is not encouraged. Start your env in Py27 and talk to it from Py35 is more sustainable.

  1. Build OpenSim yourself (since @kidzik didn't do this for us)

    • Install VC++ 2015 build tools. Should not take long

    • CMD

      • Clone the opensim conda builder, then run the build command:

        (chinese users) > set "HTTPS_PROXY=whatever_you_use"
> git clone https://github.com/opensim-org/conda-opensim.git
> cd conda-opensim
> conda build opensim --python 3.5

        Above will fail on Chinese Windows due to file encoding error. If so, don't run conda build again, as that will create a new build environment and pull the whole OpenSim from GitHub again.

        Instead, open D:\Anaconda3\conda-bld\opensim_1499279773305\work\dependencies\BTK\Utilities\Open3DMotion\src\Open3DMotion\MotionFile\Formats\MDF\FileFormatMDF.cpp and replace every occurence of the square symbol (superscript "2") with "^2".

        The file should now look like:

        //...
{ "Marker Acceleration", "m/s^2", "%.3fm/s2", "%.3f", (float)0.05},
//...

      • Run the build script generated by the build process above again:

        > cd D:\Anaconda3\conda-bld\opensim_1499279773305(actual-path-may-vary)\work
> bld.bat

        The compilation should run smoothly this time.

      • Since CMD was used to run bld.bat, you may notice that some commands, such as an cp (copy) did not run successfully. It should produce errors like that:

        (d:\Anaconda3\conda-bld\opensim_1499279773305\_b_env) Nonecp d:\Anaconda3\conda-bld\opensim_1499279773305\_b_env\Library
\simbody\bin\simbody-visualizer.exe   d:\Anaconda3\conda-bld\opensim_1499279773305\_b_env\simbody-visualizer.exe
'cp' is not recognized as an internal or external command,
operable program or batch file.`

        you can do the copy manually, though.

      • also the python setup.py install might not run smoothly:

        (d:\Anaconda3\conda-bld\opensim_1499279773305\_b_env) Nonepython setup.py install
Traceback (most recent call last):
  File "setup.py", line 8, in <module>
    execfile('opensim/version.py')
NameError: name 'execfile' is not defined

        • cause

          setup.py is written in py2, so it seems that i built the py2(default) branch of OpenSim.

        • solution

          in setup.py, replace execfile(name) with exec(open(name).read()).

        Then cd to the correct path (D:\Anaconda3\conda-bld\opensim_1499279773305\_b_env\Library\sdk\python) and manually run pip install -e .

      • Now test the OpenSim installation:

        • try import opensim in python.

        upon import opensim python may fail to find simbody's libraries.

        • solution

        add D:\Anaconda3\conda-bld\opensim_1499279773305\work\opensim_build\Release to your PATH.

  2. install osim-rl

    • CMD

      (chinese users) > set HTTPS_PROXY=whatever_you_use
> git clone https://github.com/stanfordnmbl/osim-rl.git
> cd osim-rl
> pip install -e .
> cd tests
> python test.manager.py

      You should now see the skeleton swinging!

The simulation is too slow

Update: this is not the goto choice since it introduce bias. your agent may not peform well during submission.

Visualize in environments without a graphic card (i.e. on cloud)