Test of the Analytical Jacobian

[eliafranc]

Hey there,

I have a question concerning the implementation of the analytical jacobian. I implemented the jacobian and when using the .compute_dpdq method instead of the linear estimate of the jacobian the simulation seems to work fine (and much quicker). However when I run the test-a2 to see if my implementation passes the tests, all of them turn out as failed.

My question is if it is possible to have a working analytical jacobian while not passing the test-a2? How exactly is the jacobian being tested? What is the reference (dpdq_estimated) exactly, is it your implementation of the analytical jacobian (which is exact) or is is an estimated jacobian?

Also, here is the first row of the test result, which confuses me since the error seems to be smaller than the tolerance:

test_4_AnalyticJacobian.....................................FAILED (error = 1.10134e-13 < tol = 0.0001)

Thank you for your help!

Best, Elia

[eastskykang]

hmmmm.... this is very suspicious. I will take a look and come back to you ASAP.

[yuhan-zh]

Hello! From my experience, the test [error = 1.10134e-13] is somehow not a real error, but the [PASS/ FAILED] checking should be correct. The testing program compare your estimated Jacobian and analytical Jacobian matrix, and they should be basically the same. You can edit the test.cpp file and output these two matrix for comparison. As for "the simulation seems to work fine", I am not sure if you simulate it for enough time for both estimated Jacobian and analytical Jacobian. In my case, when i didn't pass the test, using analytical Jacobian will result in crazy weird control after walking like 20s - 30s. Hope it will help.

[eastskykang]

@yuhan-zh is absolutely right. We test analytic Jacobian by comparing its value with estimated value (one computed by FD). I should have to mention this clearly in the tutorial... I will add it to frequent QnA.

You may want to double check if your estimated Jacobian is correct before you implement analytic Jacobian. (but I believe it's highly likely the case where your analytical Jacobian implementation has some error.)

Note. this is a very common approach to test if your analytic derivatives implementation is correct or not for solving numerical optimization problem with gradient-based method (since, usually, implementing analytic derivatives could be error-prone).

[eastskykang]

+p.s. and as I mentioned during the tutorial, please stick to Jacobian computed by the finite difference for your IK solver. If not, you may not get the full point from Ex.2.

[eastskykang]

@eliafranc This is the test result from your code. I see some test cases fail since some of your analytic Jacobian entries and estimated Jacobian entries are not the same within tolerance. This indicates either your analytic or estimated Jacobian is wrong.

test_4_AnalyticJacobian.....................................FAILED 
 (error = 1.10134e-13 < tol = 0.0001)
 (error = 0 < tol = 0.0001)
 (error = 0 < tol = 0.0001)
 (error = 0.1275 > tol = 0.0001)
dpdq_analytic(r, c) != dpdq_estimated(r, c)
dpdq_analytic(r, c): 0.135
dpdq_estimated(r, c): 0.2625
difference: -0.1275

 (error = 0.135 > tol = 0.0001)
dpdq_analytic(r, c) != dpdq_estimated(r, c)
dpdq_analytic(r, c): 0.135
dpdq_estimated(r, c): 0
difference: 0.135

 (error = 0.356446 > tol = 0.0001)
dpdq_analytic(r, c) != dpdq_estimated(r, c)
dpdq_analytic(r, c): 0.135
dpdq_estimated(r, c): 0.491446
difference: -0.356446

 (error = 0.14 > tol = 0.0001)
dpdq_analytic(r, c) != dpdq_estimated(r, c)
dpdq_analytic(r, c): 0.631446
dpdq_estimated(r, c): 0.491446
difference: 0.14

 (error = 0 < tol = 0.0001)
 ...
 (error = 0 < tol = 0.0001)
 (error = 1.10134e-13 < tol = 0.0001)
 (error = 0 < tol = 0.0001)
 (error = 0.491446 > tol = 0.0001)
dpdq_analytic(r, c) != dpdq_estimated(r, c)
dpdq_analytic(r, c): -0.491446
dpdq_estimated(r, c): 0
difference: -0.491446

 (error = 0.228946 > tol = 0.0001)
dpdq_analytic(r, c) != dpdq_estimated(r, c)
dpdq_analytic(r, c): -0.491446
dpdq_estimated(r, c): -0.2625
difference: -0.228946

 (error = 0.626446 > tol = 0.0001)
dpdq_analytic(r, c) != dpdq_estimated(r, c)
dpdq_analytic(r, c): -0.491446
dpdq_estimated(r, c): 0.135
difference: -0.626446

...

By the way, (error = 1.10134e-13 < tol = 0.0001) right after FAILED simply tells you the very first test case passed. There should be a line break after FAILED . I will fix the code soon for the future. (but you don't need to fix it for this assignment)

[eliafranc]

Thank you for both of you feedbacks (eastskykang @yuhan-zh)! In that case I will have to go over the implementation again and maybe simulate for longer periods of time. Thank you either way :)