ikostrikov / pytorch-a2c-ppo-acktr-gail

PyTorch implementation of Advantage Actor Critic (A2C), Proximal Policy Optimization (PPO), Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation (ACKTR) and Generative Adversarial Imitation Learning (GAIL).
MIT License
3.6k stars 829 forks source link

Specific baselines version needed for vector environments? #180

Closed mattroos closed 5 years ago

mattroos commented 5 years ago

When trying to train a Pong model using PPO, as described in the README:

python main.py --env-name "PongNoFrameskip-v4" --algo ppo --use-gae --lr 2.5e-4 --clip-param 0.1 --value-loss-coef 0.5 --num-processes 8 --num-steps 128 --num-mini-batch 4 --log-interval 1 --use-linear-lr-decay --entropy-coef 0.01

I get this error:

Logging to /tmp/openai-2019-04-06-11-16-43-417068
Traceback (most recent call last):
  File "main.py", line 196, in <module>
    main()
  File "main.py", line 42, in main
    args.gamma, args.log_dir, device, False)
  File "/home/mroos/Code/pytorch-a2c-ppo-acktr-gail/a2c_ppo_acktr/envs.py", line 91, in make_vec_envs
    envs = ShmemVecEnv(envs, context='fork')
TypeError: __init__() got an unexpected keyword argument 'context'

I have a fresh install of baselines, and the ShmemVecEnv class has no 'context' argument, as the error indicates.

class ShmemVecEnv(VecEnv):
    """
    Optimized version of SubprocVecEnv that uses shared variables to communicate observations.
    """

    def __init__(self, env_fns, spaces=None):

Is a specific version of baselines required?

ikostrikov commented 5 years ago

Wait a minute, I will try to update the code for the current version of baselines.

ikostrikov commented 5 years ago

The latest version of their github repo has the argument:

https://github.com/openai/baselines/blob/master/baselines/common/vec_env/shmem_vec_env.py#L25

Try to check your installation. Probably, python picks an old one.

mattroos commented 5 years ago

Yes, just discovered that as well. Something else on my end. Thanks.