Lesson: OpenAI Gym Installation

[gutfeeling]

Setup

1. Ubuntu 18.04 Virtual Machine in its initial state.
   • Run sudo apt-get update && sudo apt-get upgrade before recording.
2. Browser
   • https://gym.openai.com/
   • https://github.com/openai/gym#supported-systems
   • https://github.com/openai/gym/issues/809

Steps

1. In this course, I want you to start coding as soon as possible. Therefore, we will start the course by installing a Python package called Gym. Gym is a Python package that offers a practical hands-on way of learning about Reinforcement Learning. You will be able to pickup the basic concepts of Reinforcement Learning while you are coding if you use this package. So let's install it.
2. https://github.com/openai/gym#supported-systems If we look at the README file in Gym's GitHub repository, we see that Gym is supported on Linux and MacOS. Windows support is experimental. I recommend that you use Linux or MacOS for this course. If you must use Windows, I recommend using a virtual machine with a Linux installation. I have put a link in the lecture notes which can help you with setting up a virtual machine with Linux. You will be able to do most of this course using such a setup, but you might have trouble using your GPUs. Therefore, the better way is to simply use a Linux or MacOS operating system.
3. I am going to demonstrate the installation of OpenAI Gym on a Ubuntu 18.04 system. Please carry out the steps in your own computer. The steps are the same for MacOS, but the command for installing system packages is different.
4. First, update the system packages. sudo apt-get update && sudo apt-get upgrade.
5. Then create a folder called rl_course and go into that folder. mkdir rl_course && cd rl_course. You are going to do all your work for this course inside this folder.
6. Now we will install the Python package Gym.
7. https://github.com/openai/gym#supported-systems Please make sure that you have Python 3.5 or higher for this course. You can check this by typing python3 --version. As you can see, I have version .... If you have a lower Python version, please install Python 3.5 or higher.
8. https://github.com/openai/gym/issues/809 Python packages like Gym can be installed in many ways. You can use a system installation. I would discourage a system installation as this leads to all python packages used by all of your projects to be in one place. If two projects use different versions of the same package, then this can lead to problems. It's better to install Python packages in project specific sandboxes so that projects can manage their Python package independent of other projects. There are two ways of doing that. You can use a Python virtual environment. You can also use Conda. Unfortunately, Conda doesn't officially support Gym yet. Therefore, I recommend a Python virtual environment installation.
9. There are many ways to create virtual environments and manage them. I am going to use the method that is most uniform across OS and Python versions.
10. Install the virtualenv package by typing python3 -m pip install virtualenv.
11. Create the virtual environment python3 -m virtualenv venv This will create a folder called venv. This folder is a sandbox which will contain all the Python packages that we install for this course.
12. Before you start working on the project, you must activate the sandbox. You do that by typing source venv/bin/activate. This ensures that any subsequent pip install command will install the packages in the virtual environment and not in the system Python. If the command worked, you will see the name of the virtual env in the command line prompt.
13. Similarly, when you want to deactivate the sandbox, simply type deactivate in the command line. This will deactivate the virtual environment and the name will disappear from the command prompt.
14. So activate your virtual environment and update pip pip install --upgrade pip.
15. The install Gym pip install gym.

16. Check the installation with the following code

    import gym 
    
    env = gym.make("CartPole-v0")
    observation = env.reset()
    env.render()

    If the image pops up, this means your installation is successful.

    Related to

• Install gym[box2d] when we first encounter a Box2D environment in the course.
• Install gym[atari] when we first encounter an Atari environment in the course.

I have tested that the following sequence of installation steps work smoothly.

1. Install gym using pip. The classic control environments are available after this step. We are doing this in this lesson.
2. Install system level dependencies specified in https://github.com/openai/gym/blob/master/py.Dockerfile. Do this and the next step when introducing the first Box2D environment in the course.
3. Install gym[box2d] using pip. The Box2D environments are available after this step.
4. Install gym[atari] using pip. The Atari environments are available after this step. Do this when introducing the first Atari environment in the course.