code generates different result using snopt_cwrap.h and f2c interface

[hongkai-dai]

I have a simple optimization problem, and I implemented one function that uses the f2c.h, and another one using snopt_cwrap.h. They generate different output. The one with f2c is more accurate than the one with snopt_cwrap.h (as can be seen in the "Optimal" column of the print out file). The two print out files are f2c_out.txt snopt_cwrap_out.txt

The only difference in the setup part is the "Unbounded objective", and I think the function evaluation part are the same. Should I expect that snopt_cwrap.h and f2c.h produce different result? I am using SNOPT.7.6

[gnowzil]

One thing I notice is the first LU factors for each run don't seem to match.

Do you think you could provide the example? If not, can you rerun with "System information yes". That might provide some extra info.

[hongkai-dai]

Thanks Elizabeth! I added "System information yes", and here is the output file f2c_out.txt snopt_cwrap_out.txt

One thing I notice is that at iteration 0, snopt_cwrap_out.txt says the Hessian is set to scaled identity matrix, while f2c_out.txt doesn't say so. Would this be the reason?

I am writing a wrapper for SNOPT to be called through drake, and the current code that reproduce this problem is very complicated. Do you think the print out file provides enough information? If not, I will try to create a small example to reproduce the problem.

[gnowzil]

No I think that's a red herring. Extra output messages were added in the last few years. Sorry, I forgot to include some other options: Print frequency 1 Summary frequency 1 Minor print level 1 Major print level 11 System information yes

It kinda just looks like a different basis is chosen initially (thus different LU factors). Or maybe a different initial point? Note the different values in the "optimal" column at iteration 0.

[hongkai-dai]

Thanks!

Here are the print out files. snopt_cwrap_out.txt f2c_out.txt

I think the input values are the same. I checked x, xmul, xstate, all are set to 0.

[hongkai-dai]

Here is the code to reproduce the output, using snopt_cwrap.h lp.c.zip I also need to make a change to snopt/interfaces/Submakefile to compile lp.c

--- a/interfaces/Submakefile
+++ b/interfaces/Submakefile
@@ -23,7 +23,7 @@ SNOPT_C_LO   = $(SNOPT_C_FILES:%=$(API_OBJDIR)/%.lo)
 SNOPT_CPP_LO = $(SNOPT_CPP_FILES:%=$(API_OBJDIR)/%.lo)

 #-----------------------------------------------------------------------
-SNOPT_C_EX   = toyA  toyB  toyC  hs118
+SNOPT_C_EX   = toyA  toyB  toyC  hs118 lp
 SNOPT_CPP_EX = sntoya  sntoyb  sntoyc  catmixa  hs118

 SNOPT_C_EXAMPLES   = $(SNOPT_C_EX:%=$(CEX_OBJDIR)/%)

BTW, how should I build the examples in snopt/cexamples folder, such as csntoya? This example uses f2c interface. I tried the following commands

$ configure --enable-f2c --with-c --with-cxx
$ make cexamples

but I only see the executables from snopt/interfaces/cexamples; I do not see the target from snopt/cexamples.

[gnowzil]

I don't think there's a bug. I rewrote the problem in Fortran to call snopt directly and I get the same results. A lot of stuff changed internally in 2014 that is causing the differences you're seeing. I will check with my colleagues to see what they think, but they are currently out-of-office.

Out of curiosity, why do you specify the derivatives of the objective as nonlinear even though they're all constant/linear? If you formulate the problem entirely as an LP, you would "fix" this issue.

I stopped doing anything to support the f2c version years ago. I would just manually compile/link the example. --with-c and --with-cpp are not related to the f2c version.

[hongkai-dai]

I see, so I won't worry about it any more.

You are absolutely right that I should have written the objective as a linear function. I currently wrote it as a nonlinear function due to a drawback in my wrapper, that treats any cost function as nonlinear functions. I will fix that bug soon. But there are other problems with actual nonlinear cost, that I also observe that the f2c interface and snopt_cwrap interface give different results.

Apologize for my ignorance on compilation, to compile the f2c version, I tried

$ gcc -I$HOME/snopt/build/include -L$HOME/snopt/build/lib -lf2c -lsnopt7 MY_TEST_FILE.c -o MY_TEST_FILE

but the compiler complains "undefined reference to snopenappend_'", should I link to some other libraries? I thoughtsnopenappend_` is in the f2c library.

[hongkai-dai]

Sorry to bother you again, but we have ran into more problems when we switched to snopt-interface from f2c interface. Some of our problems that can be solved using f2c interface, is not solved any more with snopt-interface. Here are the outputs logs.

f2c.txt snopt_interface.txt

It seems to me that at the beginning before the major iteration, the LU factorization generates different results.

Should I add some more flags to generate more info in the output file? Should I set "System info" to Yes?

[hongkai-dai]

One thing I do not quite understand, is that in the log snopt_interface.txt, it takes 1000 major iterations, but only 13 Itns. Should Itns be the number of minor iterations? I would suppose that the number of minor iterations should be more than the number of major iterations?

[hongkai-dai]

Here are two logs with more information. f2c.txt snopt_interface.txt

[gnowzil]

Can you tell me what version/date of SNOPT you're running in the f2c problems? Your output always cuts off the output displaying that info.

[hongkai-dai]

It is snopt 7.6.1

[gnowzil]

For the f2c'd version? I didn't think we got past 7.5 with f2c. Can you check -- in file csrc/sn02lib.c, line 54 should be the line that prints the version.

[hongkai-dai]

I see, the f2c version is 7.4-12

We have several other problems, that are also solved very quickly by the f2c interface (with 7.4-12), but is not solved with 7.6.1 using c interface.

[hongkai-dai]

I write one failure case in C code, here is it. problem.c.zip To build this code, I make the following change to interfaces/Submakefile

--- a/interfaces/Submakefile
+++ b/interfaces/Submakefile
@@ -23,7 +23,7 @@ SNOPT_C_LO   = $(SNOPT_C_FILES:%=$(API_OBJDIR)/%.lo)
 SNOPT_CPP_LO = $(SNOPT_CPP_FILES:%=$(API_OBJDIR)/%.lo)

 #-----------------------------------------------------------------------
-SNOPT_C_EX   = toyA  toyB  toyC  hs118
+SNOPT_C_EX   = toyA  toyB  toyC  hs118 problem
 SNOPT_CPP_EX = sntoya  sntoyb  sntoyc  catmixa  hs118

 SNOPT_C_EXAMPLES   = $(SNOPT_C_EX:%=$(CEX_OBJDIR)/%)

When I use SNOPT 7.6.1 with C interface, this problem takes 1000 major iterations, but still cannot solve it. I could solve this problem using the f2c interface with SNOPT 7.4-12.

The problem is actually feasible. If I change the initial guess to the one in line 283 of problem.c, then the problem is solved within tens of major iterations, using 7.6.1 and 7.4-12.

We observe this performance degradation on several of our robot motion planning problems, that when we switch to SNOPT 7.6.1 with C interface, the same problem that could be solved by 7.4-12 with f2c interface is not solvable any more. Is there some parameters I should tune to make our problem more friendly to SNOPT 7.6.1?

[hongkai-dai]

We tested that problem.c code with SNOPT 7.6.1. We do not have SNOPT 7.7 on our side yet. Does SNOPT 7.7 work well on that problem?

[gnowzil]

Can you verify for me that lp.c is the same problem that is being run for the output you provided on Jul 10?

I'm looking more carefully at it and I've noticed that there seems to be an inconsistency with the constraint that has bounds (-100,40). Also, the constraints are obviously in a different order (though that shouldn't be a problem if everything is reordered correctly).

[hongkai-dai]

Thanks Elizabeth for getting back to us. lp.c is not the exact same problem as the one that generated the output on Jul 10.

But we do not worry about lp.c any more, as when we change the cost to linear form, the accuracy of the solution is good.

On the other hand, when we use SNOPT 7.6, we have difficulty to solve another problem, as mentioned in https://github.com/snopt/snopt-interface/issues/19#issuecomment-410346280. In that problem.c, SNOPT 7.6 does not converge at all. And interestingly, its iteration number doesn't increase, after the 13th iteration, while the major iteration number keeps increasing. The same problem runs find with snopt 7.4 using f2c interface. This is not the only problem that we can solve easily with snopt 7.4 f2c interface, but cannot with snopt 7.6 using C interface. How should I tune my problem to make it working well with snopt 7.6? Will snopt 7.7 solve this problem fine?

[gnowzil]

For problem.c, can I verify that you are only interested in finding a feasible point (as there's no objective)? Are the other problems that don't work the same? Also is there anything special about the two starting points that you have (or were they randomly chosen)?

[hongkai-dai]

1. Form problem.c, I am only interested in finding a feasible point, and there is no objective.
2. For the other problems that don't work, they do have objectives. These problems are much more complicated, that I cannot write a simple test case without using our Drake library.
3. There is nothing special about these two starting point. I choose them arbitrarily.

[gnowzil]

Ok thanks. We're working with problem.c. It might take us a while. The major changes after 7.4 did change with the way feasibility is handled so we'll try to figure out what's happening and if we can improve the current performance.

[hongkai-dai]

Awesome! Thanks a lot for your kind help, much appreciated.

[gnowzil]

In a very roundabout way, I found out that problem.c is able to find a feasible point (even more quickly than before) if you change the "minor optimality tolerance" to 1.0d-5 (the default is 1.0d-6).

Can you set that option and let me know if that solves the issues you're having with 7.7?

Update: I also had to turn on scaling ("scale option 2").