Small installation instructions enhancement

[rhaps0dy]

Plus minor nitpicks and likely bugs.

Also, I am working on an implementation of Async Actor-Critic, which I read somewhere you have in progress. If you're also working on it now, please tell me so we can coordinate and avoid duplicate work.

[carpedm20]

@rhaps0dy Thanks for fix. Yes I'm working on A3C and you can see the codes at https://github.com/devsisters/async-rl-tensorflow. But before finishing it, I had a hard time debugging this repo and https://github.com/devsisters/DQN-tensorflow/ to add Double Q-learning, Duelling network, and set the environment which is exactly same as Nature paper.

There is non-convergence issue at https://github.com/devsisters/DQN-tensorflow/ which contains:

• Max pooling for most recent 2 frames
• Frame skip (action repeat) for 4 times
• Terminal if life is discounted
• Duelling network
• Double Q-learning
• Give -1 reward if dead (this is not used in Nature but by adding this, I can train this model with upper features. Without this, not working for frame_skip=4 or 2 but only working for frame_skip=1)

I created multiple repos but this repo, https://github.com/carpedm20/deep-rl-tensorflow is the most cleanest and my favorite. So you are interested in implementing A3C and is there any codes you wrote since now?

[rhaps0dy]

Hello,

Well, I'm glad this is the cleanest repo. I wrote an implementation of A3C, but it works really badly. So far the code is just at my fork of this repo, https://github.com/rhaps0dy/deep-rl-tensorflow/ . If you execute the command for debugging in the README, it learns, but slowly and it doesn't get much reward. Note the stupidly small learning rate. Perhaps you can identify what is wrong?

What do you mean, a non-convergence issue? In this repo? It seems to work fine for me.