Implement VPG

[keiohta]

Policy Gradient Methods for Reinforcement Learning with Function Approximation

[keiohta]

Need to design carefully so that both on-policy and off-policy agents coexist in Trainer

[keiohta]

• REF: Likelihood Ratio Gradient Estimation for Stochastic Systems for Gaussian policy

[keiohta]

Compare followings:

• [ ] Sum of rewards version
• [ ] Reward to go version
• [ ] Reward to go + V baseline version
• [ ] Q(Action value func) version
• [x] Advantage version
• [x] GAE version #40