I tried using this sample to train on an AWS m4.16xlarge instance with Tensorflow 1.0.0 and 32 workers but it seems the model never learned anything: the agent's training result doesn't improve after 0.6 hours (over 30 minutes) while it is stated that "the agent is able to solve the same environment in 10 minutes" on the GitHub page.
The progress was monitored through TensorBoard. Please notice the example given in the GitHub was shown for the case of 16 workers, and it converges to an episode reward of 21 within 30 minutes, while for this case, with doubled number of workers and same amount of training time, the reward doesn't improve. I also took a look at the log and it seems there's no compiling error.
After spending a long time searching for solutions, I noticed there was one post mentioned successful training with older version of Tensorflow. I downgraded my tensorflow from 1.0.0 to 0.11.0 and now it trains successfully as shown in the Readme page.
Maybe there's a bug on tensorflow's side - though I thought i'd bring it up so if it's Openai gym that has some compatibility issue with then it would be beneficial to fix it.
I tried using this sample to train on an AWS m4.16xlarge instance with Tensorflow 1.0.0 and 32 workers but it seems the model never learned anything: the agent's training result doesn't improve after 0.6 hours (over 30 minutes) while it is stated that "the agent is able to solve the same environment in 10 minutes" on the GitHub page.
The progress was monitored through TensorBoard. Please notice the example given in the GitHub was shown for the case of 16 workers, and it converges to an episode reward of 21 within 30 minutes, while for this case, with doubled number of workers and same amount of training time, the reward doesn't improve. I also took a look at the log and it seems there's no compiling error.
After spending a long time searching for solutions, I noticed there was one post mentioned successful training with older version of Tensorflow. I downgraded my tensorflow from 1.0.0 to 0.11.0 and now it trains successfully as shown in the Readme page.
Maybe there's a bug on tensorflow's side - though I thought i'd bring it up so if it's Openai gym that has some compatibility issue with then it would be beneficial to fix it.