Closed witwolf closed 4 years ago
not just it fails. Trac seems does not work any more. You can assign this issue to me if you are not working on it.
In TracAlgorithm.after_train(), the action before train is the original action from rollout, which is stochastic.: https://github.com/HorizonRobotics/alf/blob/dccefccc40fe82256f554a3b633c2ec93e6aa57f/alf/algorithms/trac_algorithm.py#L140 And the action after train is also stochastic because of epsilon_greedy=1.0: https://github.com/HorizonRobotics/alf/blob/dccefccc40fe82256f554a3b633c2ec93e6aa57f/alf/algorithms/trac_algorithm.py#L220-L221 This two stochasticity make it impossible to reduce the change. So TrustedUpdater.adjust_step() fails after trying 100 steps.
not just it fails. Trac seems does not work any more. You can assign this issue to me if you are not working on it.
Yes, it does not work , and i have made a few change on https://github.com/HorizonRobotics/alf/commit/b521c6aa0d2f09ff905355c1926b38eab94bd321
\frac{\sum^T\sum^B\sqrt{D}}{B*N}
instead of \sqrt{\frac{\sum^T\sum^BD}{B*N}}
and it still can not work with ddpg
not just it fails. Trac seems does not work any more. You can assign this issue to me if you are not working on it.
Yes, it does not work , and i have made a few change on b521c6a
- using predicted action (noiseless action) when calc distance for old and new policy
- calc total dist as
\frac{\sum^T\sum^B\sqrt{D}}{B*N}
instead of\sqrt{\frac{\sum^T\sum^BD}{B*N}}
and it still can not work with ddpg
It seems that you only fixed the first stochasticity.
Fixed by #321
when test with trac_ddpg_pendulum
get error msg below, still investagte on it