cobyla returns last evaluated function which might not be minimum

[ericjster]

COBYLA returns the wrong value for minimum. It returns the last evaluated function value, but that last value might not be the return value. Attached is an example of minimizing a test function ("Powell's Badly Scaled Function"). NEWUOA and BOBYQA do well.

The output shows the minima is found (close to 0.0), but then some additional evaluations are done and it seems to forget the best value. ... myfunc: x: [ 5.39347657e+00 2.17974583e-05] , val: 0.0308692676819 myfunc: x: [ 5.39307750e+00 -2.79444985e-04] , val: 258.266926113 myfunc: x: [ 5.39385142e+00 -3.09088486e-04] , val: 312.291607364 optimum at 5.39385141967 -0.000309088485691 minimum value = 312.291607364 result code = 4

nlopt_cobyla_bug.py.txt

[jschueller]

confirmed, this is a bug in cobyla.c, appears in the original powell version as well.

@stevengj, maybe the algorithm does not check for optimality for each feasible point of the simplex after it's evaluation with calcfc (it does not go to L620 but "loops"). Maybe something could be added as you did around line 600 ? The code is quite hard to follow because of these gotos, I hope you still know the code : )

[E-B41]

I think we have problems with the same bug. The algorithm does not always return the optimum set of variables + these variables might violate given constraints. Looking at the course of optimization variables there are sets, that are okay. This seems also be related to Issue #182 .

Is there any progress on this bug? @jschueller is it correct, that the OpenTURNS implementation saves the optimization progress and uses it to return the correct result?

[markusbrugger]

We have stumbled across this bug when finding the minimum of a ratio of polynomials with several inequality constraints using nlopt's COBYLA algorithm. We are quite happy with the nlopt library, because of its neat and plain integration as a C library into our C program. However, the above-mentioned bug is still present in the current version 2.6.2. Our abject question hence is, why there has been no progress on this issue during the past? Surely, a work-around is to keep track of the optimization history manually (what we actually do now) and check in the end if COBYLA returns the correct values, otherwise take the really correct values from the external array that keeps the best result of the objective function together with the corresponding, non-violating parameter set. This procedure seems and is in fact clumsy, especially considering the fact that many users might not be aware of the bug, because their final values can rarely be called "suspicious". In our case, the bug was straightforward to detect, because before COBYLA takes over the steering wheel, a first round of a manual optimization takes place, whose optimized parameters are then handed over to COBYLA as valid starting values. In the end, COBYLA from time to time ends up with a score of, say, -20.0, although it started with, say, 2.5 (the larger the better in this case). We think, fixing this bug should not be too demanding (see also jschueller's comment from 2016 above), but it would prevent the scientific world from a bunch of false-positive/negative results due to a tiny, but nasty bug in the return behaviour of COBYLA. We are sure that fixing this bug would greatly improve the nlopt library and substantially contributes to its trustworthyness.

We cordially hope for some comments on this topic and would also offer help to find a solution to the problem!

[zaikunzhang]

We have stumbled across this bug when finding the minimum of a ratio of polynomials with several inequality constraints using nlopt's COBYLA algorithm. We are quite happy with the nlopt library, because of its neat and plain integration as a C library into our C program. However, the above-mentioned bug is still present in the current version 2.6.2. Our abject question hence is, why there has been no progress on this issue during the past? Surely, a work-around is to keep track of the optimization history manually (what we actually do now) and check in the end if COBYLA returns the correct values, otherwise take the really correct values from the external array that keeps the best result of the objective function together with the corresponding, non-violating parameter set. This procedure seems and is in fact clumsy, especially considering the fact that many users might not be aware of the bug, because their final values can rarely be called "suspicious". In our case, the bug was straightforward to detect, because before COBYLA takes over the steering wheel, a first round of a manual optimization takes place, whose optimized parameters are then handed over to COBYLA as valid starting values. In the end, COBYLA from time to time ends up with a score of, say, -20.0, although it started with, say, 2.5 (the larger the better in this case). We think, fixing this bug should not be too demanding (see also jschueller's comment from 2016 above), but it would prevent the scientific world from a bunch of false-positive/negative results due to a tiny, but nasty bug in the return behaviour of COBYLA. We are sure that fixing this bug would greatly improve the nlopt library and substantially contributes to its trustworthyness.

We cordially hope for some comments on this topic and would also offer help to find a solution to the problem!

Hello @markusbrugger and others,

Sorry for revitalizing this old discussion.

I am the person that is responsible for maintaining Professor Powell's Fortran code, including COBYLA (however, I am not involved in NLopt).

Is this bug still bothering you? It has been fixed in the PRIMA version of Powell's code.

I understand that you need to use COBYLA in C. Providing a C interface or C implementation is on the TODO list of the PRIMA project, but it will not be done soon. Therefore, you may need to code your own interface if you are interested in trying the PRIMA version of COBYLA.

It might be helpful to mention that @sanketr is developing C/C++ interfaces for the PRIMA solvers according to an issue raised on the PRIMA repo.

PRIMA has fixed some other serious bugs in the Fortran 77 code, which may crash your computer or make it trapped in a dead loop.

In addition to bug fixes, the performance of the PRIMA code is much better than Powell's Fortran 77 code in terms of the number of function evaluations, which normally represent the major computational cost for the problems handled by these solvers.

This procedure seems and is in fact clumsy, especially considering the fact that many users might not be aware of the bug, because their final values can rarely be called "suspicious".

We think, fixing this bug should not be too demanding (see also jschueller's comment from 2016 above), but it would prevent the scientific world from a bunch of false-positive/negative results due to a tiny, but nasty bug in the return behaviour of COBYLA.

Even though it is indeed nontrivial to fix the bug completely, I entirely agree with your point that it is crucial to get the bug(s) fixed in Powell's solvers, because they might be among the most widely used optimization solvers in the scientific/engineering world (see, e.g., Section 1 of a recent paper on Powell's solvers as well as the Google searches of COBYLA and BOBYQA.

Tell me if you need more information on Powell's solvers and PRIMA. It is my responsibility to help, although I can only help from the Fortran side for the moment.

Thank you.

Zaikun Department of Applied Mathematics The Hong Kong Polytechnic University