search

ikostrikov / pytorch-a2c-ppo-acktr-gail

PyTorch implementation of Advantage Actor Critic (A2C), Proximal Policy Optimization (PPO), Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation (ACKTR) and Generative Adversarial Imitation Learning (GAIL).
MIT License
3.59k stars 829 forks source link

Add Dockerfile #9

Open AwokeKnowing opened 7 years ago

AwokeKnowing commented 7 years ago

Can you list the exact commands you would run to install the dependencies? Then we can make a Dockerfile so people can run it as a docker container on any computer/server without installing anything.

ikostrikov commented 7 years ago

I added requirements to the readme file. Let me know if anything is missing.

AwokeKnowing commented 7 years ago

I haven't got a chance to fine tune it, but for now, I have it working. Based off official pytorch docker image, I just had to add the extra dependencies open AI gym mentions on their install page.

FROM pytorch/pytorch:v0.2
COPY . .
RUN apt-get update -qq && apt-get install -y --no-install-recommends \
 python-numpy python-dev cmake zlib1g-dev libjpeg-dev xvfb libav-tools \
 xorg-dev python-opengl libboost-all-dev libsdl2-dev swig \
 && apt-get clean && rm -rf /var/lib/apt/lists/*
RUN git clone https://github.com/openai/baselines.git && cd baselines && pip install -e .
RUN pip install -r requirements.txt
CMD ['/bin/sh']

So steps are:

  1. Clone this repo
  2. Download your trained models (see link in readme)
  3. Add the above into repo as file named Dockerfile
  4. Build the image docker build -t "pytorch-a2c-ppo-acktr" ./
  5. Run the docker image. Here we 'enjoy' pong:
nvidia-docker run --rm -it -v "$(pwd)/trained_models":/workspace/trained_models \
    --user=$(id -u) \
    --env="DISPLAY" \
    --volume="/tmp/.X11-unix:/tmp/.X11-unix:rw" \
    awokeknowing/pytorch-a2c-ppo-acktr \
    python enjoy.py --algo acktr --load-dir trained_models/acktr --env-name "PongNoFrameskip-v4" --num-stack 4

Again, you might consider having the built image at docker hub so that you could get your 'volunteers' up and running with just cloning the repo and running a script like bash volunteer.sh (assuming they have nvidia-docker installed)

Also, note that it works with regular 'docker' with identical command. nvidia-docker just makes sure gpu is available, which you'd want for training.

I'll update as I do some more testing.