Open maystroh opened 5 years ago
I've fixed it.. It was just that I used the wrong method. I should use this method
env = make_unity_env(env_id, args.num_env or 1, args.visual_obs)
instead of env = make_vec_env(env_id, env_type, args.num_env or 1, seed, reward_scale=args.reward_scale)
def build_env(args):
ncpu = multiprocessing.cpu_count()
if sys.platform == 'darwin': ncpu //= 2
nenv = args.num_env or ncpu
alg = args.alg
rank = MPI.COMM_WORLD.Get_rank() if MPI else 0
seed = args.seed
env_type, env_id = get_env_type(args.env)
if env_type == 'atari':
..
elif env_type == 'retro':
..
elif env_type == 'unity':
get_session(tf.ConfigProto(allow_soft_placement=True,
intra_op_parallelism_threads=1,
inter_op_parallelism_threads=1))
# env = make_vec_env(env_id, env_type, args.num_env or 1, seed, reward_scale=args.reward_scale)
env = make_unity_env(env_id, args.num_env or 1, args.visual_obs)
# env = VecNormalize(env)
Hi @maystroh - I'm glad you figured it out.
It would be good to get your feedback as at some point, it would be good to fold the baselines capabilities back into ml-agents.
For me, I was exploring baselines to see if I could speed up training over ml-agents (leverage the gpu etc) - however, I found that
I've also been trying to get HER working, however, the baselines HER code is quite deeply coupled with MPI and Mujoco - so I'm thinking it may be faster to try implementing DDQN = HER into ml-agents
Actually, I have the same goal: exploring the baselines to double check if their PPO implementation is really optimized for GPU. Since I'm working with visual observation, the trainings are expected to be faster with GPU no? Just to verify your outcomes: gpu was slower than cpu for both vector and visual observations?
So far, I'm working with only one agent but will try double check this case once I finish what I have to accomplish. I will update this thread with whatever interesting info I find during my exploration.
I have not tried visual observations yet - so I'm interested to know how this works out!!
Yes, I can confirm that CPU was fast than GPU in my tests. My state/action spaces are small and also my buffer size is small(ish) so this may be contributing to why that was the case. Note: I'm using an optimized version of tensorflow - here are some links:
sudo pip install https://github.com/vearutop/builds/releases/download/tensorflow-1.7.0-cp36-cp36m-macosx_10_13_x86_64.whl/tensorflow-1.7.0-cp36-cp36m-macosx_10_13_x86_64.whl
sudo pip -h https://github.com/vearutop/builds/releases/download/tensorflow-1.7.0-cp36-cp36m-macosx_10_13_x86_64.whl/tensorflow-1.7.0-cp36-cp36m-macosx_10_13_x86_64.whl
I'm trying to use your code to run the baselines algos of openai with a unity3D environment. Here is the command I'm using to launch the training:
python -m baselines.run_unity --alg=ppo2 --env=./envs/env.x86_64 --num_timesteps=1e6 --save_path=./models/test
Here is the problem I'm getting:
The problem is coming from the function
ortho_init
inutils.py
. Please check below for more details.I just wanted to report it here in case I can get some help ASAP. I will continue my inquiries and will post more details once I fix it.