Compute returns and advantage should not be a method of RolloutBuffer. This should be implemented as a separate function or Class to allow for flexibility in extension of this.
There's the usual computation of reward on a trajectory but as an example we also have Generalized Advantage Estimation. GAE could be a separate function/inherting class of the first case.
This, in general, allows the classes to be more composable.
Compute returns and advantage should not be a method of
RolloutBuffer
. This should be implemented as a separate function or Class to allow for flexibility in extension of this.There's the usual computation of reward on a trajectory but as an example we also have Generalized Advantage Estimation. GAE could be a separate function/inherting class of the first case.
This, in general, allows the classes to be more composable.