Failed to find finite solution using black-box derivative

[dtch1997]

A problem has emerged; apparently the gradient is needed somehow to be able to solve the optimal control w.r.t the derivative. When training inverted pendulum:

With gradients:

/home/daniel/Documents/github/IsaacGymEnvs/third-party/neural_clbf/neural_clbf/systems/control_affine_system.py(147)compute_linearized_controller() -> K_np = lqr(A, B, Q, R (Pdb) p (A, B, Q, R) (array([[1. , 0.05 ], [0.49033251, 0.9995 ]]), array([[0. ], [0.05]], dtype=float32), array([[1., 0.], [0., 1.]]), array([[1.]]))

/home/daniel/Documents/github/IsaacGymEnvs/third-party/neural_clbf/neural_clbf/systems/control_affine_system.py(149)compute_linearized_controller() -> self.K = torch.tensor(K_np) (Pdb) p K_np array([[18.3127739, 5.8956387]])

Without gradients:

/home/daniel/Documents/github/IsaacGymEnvs/third-party/neural_clbf/neural_clbf/systems/control_affine_system.py(147)compute_linearized_controller() -> K_np = lqr(A, B, Q, R) (Pdb) p (A, B, Q, R) (array([[1., 0.], [0., 1.]]), array([[0. ], [0.05]], dtype=float32), array([[1., 0.], [0., 1.]]), array([[1.]])) (Pdb) continue ... numpy.linalg.LinAlgError: Failed to find a finite solution.

[dtch1997]

After some inspection, I believe this is because the method compute_A_matrix requires gradients in the form of jacobian. However, I could plausibly do something similar to what I did for the contraction metrics and replace that with a black-box gradient

[dtch1997]

Tentative solution implemented in #7 , we'll see how it goes

[dtch1997]

I think it's fixed. Closing for now