RobotLocomotion / drake

Model-based design and verification for robotics.
https://drake.mit.edu
Other
3.33k stars 1.26k forks source link

Assess the accuracy of Gurobi in cone projection tests #16887

Open amcastro-tri opened 2 years ago

amcastro-tri commented 2 years ago

I modified sap_friction_cone_constraint_test.cc to use Gurobi in order to assess its accuracy for these simple (3 variables) projection tests. The unit test is there to verify the analytical solution to these projections using a numerical computation. In master we use SCS since, even if slow when compared to Gurobi for large problems, SCS does great in these very small test cases. It also does well when we tighten the tolerances to get something at least within single floating point precision to compare against.

Gurobi in the other hand performs terribly. The solutions present (relative) errors in the order of 1e-4 and higher. Tightening the QCP tolerance improves the solution in some cases, but surprisingly, it makes things worse for some others! This is very surprising considering how well conditioned these problems are when compared to real contact problems as the ones we solve in practice with SAP, not to even mention this is a single constraint, no dynamics!

Reproducing this

Here is the branch (a single commit) with these small changes to master to run with Gurobi and log some useful information. I documented the test cases and tolerances used.

Reproducing the failures is very simple, just run: bazel run --config gurobi multibody/contact_solvers/sap:sap_friction_cone_constraint_test

The documentation in the test describes the problem setup in good detail.

cc'ing @RussTedrake since we talked about this sometime last week.

hongkai-dai commented 2 years ago

I wrote a gurobi native code (without using Drake) and the problem pertains. The accuracy is low (error >1E-4) in the default setting. The problem is that the solution should lie exactly on the boundary of the friction cone, but the gurobi solution has a small margin to the boundary (margin ~ 1E-6). I tightened BarQCPConvTol to 1E-9 and then the solution becomes quite accurate.

Unfortunately in sap_friction_cone_constraint_test tightening BarQCPConvTol doesn't produce more accurate result through Drake's MathematicalProgram. I think when Drake parses a Lorentz cone constraint it did it differently from how Gurobi parses the constraint internally. I will look more into this issue.

amcastro-tri commented 2 years ago

Thanks @hongkai-dai! great to hear you are making progress.

The problem is that the solution should lie exactly on the boundary of the friction cone

I remember seeing accuracy problems with SapFrictionConeConstraint_RegionI, where the input vectors are well within the friction cone. Is that consistent with your observations?

hongkai-dai commented 2 years ago

I remember seeing accuracy problems with SapFrictionConeConstraint_RegionI, where the input vectors are well within the friction cone. Is that consistent with your observations?

It's the similar behavior in both Region I and Region II. In my gurobi native code, the default setting is very inaccurate. If I decrease BarQCPConvTol then the solution gets much more accurate. With the Drake code, reducing BarQCPConvTol doesn't improve the solution accuracy, in fact it increases the solution error. I think there is some discrepancy in how Drake handles LorentzConeConstraint versus how gurobi parses it.

amcastro-tri commented 2 years ago

well, that's actually great news! you got the two codes to behave differently. It'd seem you are on the right track

hongkai-dai commented 2 years ago

FTR, I wrote the problem in gurobi native C code, and saw some interesting behaviors. Unfortunately there is ambiguity in how gurobi handles the Lorentz cone constraint. I asked this question on gurobi website https://support.gurobi.com/hc/en-us/community/posts/360068031552/comments/5244198105233