Open R1ckF opened 6 years ago
Hi @R1ckF! The continuous version of acktr have not gotten refactored yet (currently in the works); so you'll need to run it via python -m baselines.acktr.run_mujoco --env=Ant-v2
(for instance with Ant environment).
Hi @pzhokhov and thank you for your answer!
I just discovered this repository about 2 weeks ago and I like it a lot. Great work. I also think the refactoring is good idea, makes the code more accessible for everybody and easier to run.
I am not fully understanding everything yet (new to RL), but I did notice some discrepancies between the discrete and the continuous algorithm, so I have a few follow up questions: -I notice that ACKTR discrete and continuous print very different outputs to the console. Is this purely a cosmetic change or is the code also fundamentally different? -From the paper "Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation" it is mentioned that distributed simulation reduces sample efficiency. From reading the code it looks like ACKTR discrete can be run in multiple environments simultaneously with the "--num_env" flag. For ACKTR continous this is not the case. Is this correct? If so why is this?
Thanks again!
Hi @R1ckF ! Sorry about the lag! Short story is this - the version of ACKTR with continuous action spaces uses entirely different code as a version for discrete action spaces (except for KfacOptimizer, which both use). There are a few tweaks to the continuous version (having optimizer with separate hyperparameters for value function approximator, weight decay etc) which are implemented in the continuous version. Given that I am not the original author of the code, I cannot speak confidently of the relative role of each; however, I have opened a PR to unite discrete and continuous versions, here: https://github.com/openai/baselines/pull/560 - hopefully there will be some input from the authors. If you want to give a unified version a spin, feel free to check out the branch from the PR and try it.
Thx @pzhokhov for your response! I'll definitely check it out.
Hi all,
When I try to run a Mujoco environment with ACKTR algorithm it doesn't work. Here is the full log:
I think its because ACKTR is divided in a continous and discrete algorithm and it seems as only the discrete algorithm is loaded in baselines.acktr.acktr.py
from baselines.acktr.acktr_disc import *
It does not load acktr_cont...
When I change the script to load acktr_cont and not acktr_disc other errors occur so I am thinking there should be an easier solution? What is the correct way of running Mujoco with acktr algorithm?
Thanks in advance!