danielsuo
commented
3 years ago Farnaz:
Hello! Thanks for checking in. The implementation in deluca is a sample implementation and not in line with the paper you mentioned. I asked one of the authors and there is not a publicly available implementation.
Apologies!
Daniel
On Wed, Aug 25, 2021 at 5:40 AM FarnazAdib @.***> wrote:
Hi
Thanks for providing this interesting package.
I am trying to test drc on a simple setup and I notice that the current implementation of drc does not work. I mean when I try it for a simple partially observable linear system with A = np.array([[1.0 0.95], [0.0, -0.9]]), B = np.array([[0.0], [1.0]]) C = np.array([[1.0, 0]]) Q , R = I gaussian process noise, zero observation noise which is open loop stable, the controller acts like a zero controller. I tried to get a different response by setting the hyperparameters but they are mostly the same. Then I looked at the implementation at the deluca github and I noticed that the counterfactual cost is not defined correctly (if I am not wrong). According to Algorithm 1 in [1], we need to use M_t to compute y_t (which depends on the previous controls (u) using again Mt) but in the implementation, the previous controls based on M{t-i} are used. Anyway, I implemented the algorithm using M_t but what I get after the simulation is either close to zero control or an unstable one.
I was wondering if you have any code example for the DRC algorithm that works? [1] Simchowitz, Max and Singh, Karan and Hazan, Elad, "Improper learning for non-stochastic control", COLT 2020.
Thanks a lot, Sincerely, Farnaz
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/google/deluca/issues/47, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAURLVVEH77B7ELJZIFOV4LT6S3ABANCNFSM5CYVHC6A .
FarnazAdib
Hi
Thanks for providing this interesting package.
I am trying to test drc on a simple setup and I notice that the current implementation of drc does not work. I mean when I try it for a simple partially observable linear system with A = np.array([[1.0 0.95], [0.0, -0.9]]), B = np.array([[0.0], [1.0]]) C = np.array([[1.0, 0]]) Q , R = I gaussian process noise, zero observation noise which is open loop stable, the controller acts like a zero controller. I tried to get a different response by setting the hyperparameters but they are mostly the same. Then I looked at the implementation at the deluca github and I noticed that the counterfactual cost is not defined correctly (if I am not wrong). According to Algorithm 1 in [1], we need to use M_t to compute y_t (which depends on the previous controls (u) using again Mt) but in the implementation, the previous controls based on M{t-i} are used. Anyway, I implemented the algorithm using M_t but what I get after the simulation is either close to zero control or an unstable one.
I was wondering if you have any code example for the DRC algorithm that works? [1] Simchowitz, Max and Singh, Karan and Hazan, Elad, "Improper learning for non-stochastic control", COLT 2020.
Thanks a lot, Sincerely, Farnaz