Set/expose qpOASES's setToReliable option?

traversaro commented 3 years ago

Summary

The qpOASES QP solver with default options sometimes mark as unfeasible problems that are feasible, see https://github.com/coin-or/qpOASES/issues/85 . However, from https://github.com/coin-or/qpOASES/issues/85#issuecomment-778382108 and https://raw.githubusercontent.com/coin-or/qpOASES/master/doc/manual.pdf page 30, it seems that there is a setToReliable option that should make the QP solver more reliable and the expense of computational speed.

Motivation

If this option is present, could it make sense to set them in https://github.com/robotology/wb-toolbox/blob/master/toolbox/library/src/QpOases.cpp#L320 or expose them?

Additional context

Related issues:

https://github.com/robotology/robotology-superbuild/issues/239

traversaro commented 3 years ago

This issue may be interesting for anyone using the WB-Toolbox for solving QP problems. cc @nunoguedelha @diegoferigo @gabrielenava @Giulero @lrapetti @Yeshasvitvs

diegoferigo commented 3 years ago

From my side this sounds good. I'd recommend current users that have the qpOASES block in their Simulink Models to try this change. The modification should be straightforward as @traversaro pointed out:

https://github.com/robotology/wb-toolbox/blob/99156c73c6b651502317722dc908ba7a9ba5f4ce/toolbox/library/src/QpOases.cpp#L320-L322

qpOASES::Options problemOptions; 
problemOptions.setToReliable();  // < -- HERE
pImpl->sqProblem->setOptions(problemOptions);

traversaro commented 3 years ago

This issue may be interesting for anyone using the WB-Toolbox for solving QP problems. cc @nunoguedelha @diegoferigo @gabrielenava @Giulero @lrapetti @Yeshasvitvs

I think I forgot @CarlottaSartore @VenusPasandi @valentino-razza in this list, sorry for that.

For avoiding that, let's tag @robotology/iit-dynamic-interaction-control : if anyone arrived here with this mention and never had to solve a Quadratic Programming problem in Simulink using WB-Toolbox's QpOases block, feel free to ignore this mention. If instead you are using that block and you are experiencing strange QP failures, this may be due to the options that we are using for the qpOASES solver, and fixing them may be relatively easy, just trying the modification suggested by @diegoferigo in https://github.com/robotology/wb-toolbox/issues/202#issuecomment-779257071 .

Note that it probably still make sense to eventually investigate https://github.com/robotology/wb-toolbox/issues/180, but this modification of qpOASES is probably much faster (it can be done in ~10 minutes).

lrapetti commented 3 years ago

I did a preliminary test switching to this option, everything compiled fine and the following is the output of the profiler:

QP not failing

In this case the QP is not failing with Default option, and it is not failing as well with Reliable:

Default time = 0.224/2106 = 0.00010636277
Reliable time = 0.230/2156 = 0.00010667903

As computational time, the difference seems to be small

QP failing

I tested also in a configuration where the QP was failing with Default option, and it was failing with Reliable as well (note that I am not sure in this specific case a solution is feasible)

Default time = 0.449/2090 = 0.00021483253
Reliable time = 0.485/2118 = 0.00022898961

traversaro commented 3 years ago

Thanks @lrapetti ! What is the size of the QP problem in your case? For a problem that in the past we found that it was not working in qpOASES even if it was feasible, you can find it in the .mat file attached to the failing example in https://github.com/coin-or/qpOASES/issues/85#issuecomment-590035533 .

lrapetti commented 3 years ago

Thanks @lrapetti ! What is the size of the QP problem in your case?

Hessian: 48x48 Inequality constraint matrix: 76x48

traversaro commented 3 years ago

Cool, I think that is definitely is "big" w.r.t. to classical uses of the QpOases block in WB-Toolbox. At this point, if there is no particular performance problems, I think it make sense to enable the setReliable mode by default.

lrapetti commented 3 years ago

Inequality constraint matrix: 76x48

Actually 152x48, but with half dummy lines 0≤0

S-Dafarra commented 3 years ago

Inequality constraint matrix: 76x48

Actually 152x48, but with half dummy lines 0≤0

I would remove those. It screws up the detection of the active contacts and the constraint matrix gets singular.

lrapetti commented 3 years ago

Inequality constraint matrix: 76x48

Actually 152x48, but with half dummy lines 0≤0

I would remove those. It screws up the detection of the active contacts and the constraint matrix gets singular.

sure, I was forced to do it in order to have fixed-size signals entering the block (alternative would be instantiating multiple QP blocks, but for the time being I was not experiencing particular problems with the dummy constraints).

Cool, I think that is definitely is "big" w.r.t. to classical uses of the QpOases block in WB-Toolbox. At this point, if there is no particular performance problems, I think it make sense to enable the setReliable mode by default.

Anyway, I know what I will say is a bit vague, but today I saw some "strange" and "jumpy" output from the QP when setReliable was used, on some other code (i.e. not the one I was refering to in https://github.com/robotology/wb-toolbox/issues/202#issuecomment-784011217). Switching back to setToDefault() changed the behaviour back to a normal one.

I will try to document more this observation repeating the same scenario, but I just want to say we probably should test further before enabling setReliable as default.

traversaro commented 3 years ago

Thanks @lrapetti ! If you are able to log the time series of QP instances (i.e. cost matrices and constraint matrices) it would be useful also for testing them on OSQP.

nunoguedelha commented 3 years ago

Side note: on this simulation with feet contact handling, having dummy lines with zeros was causing the QP to fail. For handling toggling contact constraints (feet touching or not the ground), I had to toggle the boundaries of the QP problem variable.

nunoguedelha commented 3 years ago

This problem was not occurring with the OSQP (bindings).

traversaro commented 3 years ago

cc @CarlottaSartore another quick check that we could do is enable this option. @lrapetti do you happen to still have around the branch with the option changed?

lrapetti commented 3 years ago

cc @CarlottaSartore another quick check that we could do is enable this option. @lrapetti do you happen to still have around the branch with the option changed?

No, but it was set just by adding one line as @diegoferigo was pointing above (https://github.com/robotology/wb-toolbox/issues/202#issuecomment-779257071)

lrapetti commented 3 years ago

Today, I have tried again to set that option (moving to the branch in https://github.com/robotology/wb-toolbox/pull/208) on a "stable" simulink model, and I didn't encounter any problem.

In particular, I have tested again a "big" model with 3 qpOASES blocks (two solving the same problem, and one solving a larger problem), and there seems to be no computational time increase (it is actually taking less time on reliable option). Here are the statistics (qpOASES blocks are those named QP solver)

Default

not_reliable

Reliable

set_reliable reliable_2 (2)

robotology / wb-toolbox