What's the best cross-platform preconfigured setup for working with OpenAI Gym, Keras and Q-learning?

[insearchofanswers87]

I am trying to put together a Q-Learning reinforcement-learning example for students using OpenAI Gym with Keras--hopefully from a Jupyter Notebook running in Jupyter Lab. I'd prefer a complete environment that can be installed on Windows, Linux and macOS so they don't have to worry about configuration issues.

There does not appear to be a prebuilt official OpenAI Gym Docker container with all the dependencies.

I tried using OpenAI Gym's dockerfile to build a container, but killed the build when I realized it had already downloaded 50GB with no indication of how much longer it would go. I can't expect students to download 50GB+ for a small example.

I tried installing OpenAI gym and its other dependencies into the Jupyter team's jupyter/tensorflow-notebook Docker container but was unsuccessful.

I poked around online and found other Docker containers preconfigured with Jupyter, OpenAI Gym, Keras and Tensorflow. Miscellaneous folks have posted them. I tried the one I found that had everything I needed, but the execution was so slow that it would take forever to train a model.

Does anyone have recommendations on a proper environment that students could use locally on their own systems?

[bionicles]

try Anaconda https://conda.io/docs/user-guide/install/index.html Just make a list of the required steps and give the students the list of steps

here's an example of a setup guide for a different purpose: https://medium.com/bit-pharma/how-do-you-set-up-ubuntu-16-04-rigs-for-deep-bio-molecular-reinforcement-learning-5c7ae771f26a (doesn't include gym setup guide)

[bionicles]

google colab can probably make your life easier: https://medium.com/@kaleajit27/reinforcement-learning-on-google-colab-9cb2e1ef51e

and Sagemaker on AWS also could make life easier

[insearchofanswers87]

Thank you @bionicles! I'll check these out tonight.

[insearchofanswers87]

So I did not have luck with these, but as a result of them and some additional research, I discovered why TensorFlow was not working for me--Python 3.7.

As a result, Keras could not find TensorFlow—even though pip said it installed properly.

So I set up a new TensorFlow/Python 3.6.7/Anaconda environment.

When I tried running deep q-learning Keras code using IPython, I still had the same issue with TensorFlow. This turned out to be because I did not install IPython into the new environment—I assumed it was installed with anaconda, but it was not. When Anaconda can’t find IPython in the currently activated environment, it defaults to the version from the base environment, which uses Python 3.7 on my system.

Once I discovered these issues, I was able to use the following commands to get an environment up and running with OpenAI Gym on both Mac and Windows--I believe this will also work on Linux, but I have not tried it yet:

conda create -n tensorflow_env tensorflow source activate tensorflow_env conda install -c anaconda ipython pip install gym

I am now using Keras from TensorFlow’s tensorflow.keras modules, rather than installing Keras separately—I have not yet tested whether the separate Keras install was a part of my prior issues.

[shuruiz]

Anaconda/miniconda is the most efficient platform I have ever used so far. I will recommend you use python 3.6+ to work on it.

Also, jupyter notebook is a good tool for you to visualize you results step by step.

[christopherhesse]

OpenAI Gym does not officially support windows, but seems to work there. As others have mentioned, miniconda is a good start for getting things to work. You can also create an environment.yaml file to make it easy for people to get the correct environment setup: https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#creating-an-environment-from-an-environment-yml-file

If colab works for your use case though, that would be pretty amazing for ease-of-use.