Hey!
I'm a bit confused about why in the code to compute advantages, the previous advantage value is being set to the value of the first env's advantage from the previous time step, ie advantages[i, 0]
(assuming that advantages are structured in dimension/size as (time_steps X num_envs X 1 ))
Hey! I'm a bit confused about why in the code to compute advantages, the previous advantage value is being set to the value of the first env's advantage from the previous time step, ie advantages[i, 0] (assuming that advantages are structured in dimension/size as (time_steps X num_envs X 1 ))
https://github.com/Khrylx/PyTorch-RL/blob/15b574f5d52f5eeab6917c90c17e8739578f3d96/core/common.py#L17 Could you link the source for the equations for this whole function? Thanks! Gunshi