Closed moorepants closed 9 years ago
Here is a quick comparison of the solvers I have compiled for the pendulum id example:
(inverted-pendulum-id)moorepants@moorepants-2170p:examples(ipopt-options)$ ipython --pdb
Python 2.7.8 |Continuum Analytics, Inc.| (default, Aug 21 2014, 18:22:21)
Type "copyright", "credits" or "license" for more information.
IPython 2.3.0 -- An enhanced Interactive Python.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://binstar.org
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.
In [1]: import vyasarayani2011
In [2]: solvers = ['mumps', 'ma27', 'ma57', 'ma77', 'ma86', 'ma97']
In [3]: for solver in solvers:
...: vyasarayani2011.main('sysid', solver)
...:
******************************************************************************
This program contains Ipopt, a library for large-scale nonlinear optimization.
Ipopt is released as open source code under the Eclipse Public License (EPL).
For more information visit http://projects.coin-or.org/Ipopt
******************************************************************************
This is Ipopt version 3.11, running with linear solver mumps.
NOTE: Other linear solvers might be more efficient (see Ipopt documentation).
Number of nonzeros in equality constraint Jacobian...: 49990
Number of nonzeros in inequality constraint Jacobian.: 0
Number of nonzeros in Lagrangian Hessian.............: 0
Total number of variables............................: 10001
variables with only lower bounds: 0
variables with lower and upper bounds: 0
variables with only upper bounds: 0
Total number of equality constraints.................: 9998
Total number of inequality constraints...............: 0
inequality constraints with only lower bounds: 0
inequality constraints with lower and upper bounds: 0
inequality constraints with only upper bounds: 0
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
0 0.0000000e+00 6.30e+01 0.00e+00 0.0 0.00e+00 - 0.00e+00 0.00e+00 0
1 5.0242753e-01 5.47e+00 1.88e+02 -11.0 2.48e+01 - 1.00e+00 1.00e+00h 1
2 4.7533888e-01 7.66e-01 4.12e+01 -11.0 5.10e+00 - 1.00e+00 1.00e+00h 1
3 4.5749124e-01 1.02e-02 3.98e+00 -11.0 2.19e-01 - 1.00e+00 1.00e+00h 1
4 4.5635391e-01 5.60e-06 6.03e-03 -11.0 6.03e-03 - 1.00e+00 1.00e+00h 1
5 3.4576098e-01 6.06e-02 2.07e-01 -11.0 6.53e-01 - 1.00e+00 1.00e+00f 1
6 1.2453463e-01 1.73e-02 7.75e-02 -11.0 2.94e-01 - 1.00e+00 1.00e+00h 1
7 1.2138022e-01 6.36e-04 1.91e-02 -11.0 5.00e-02 - 1.00e+00 1.00e+00h 1
8 1.2086662e-01 3.06e-05 7.52e-03 -11.0 1.57e-02 - 1.00e+00 1.00e+00h 1
9 1.2085149e-01 1.09e-06 1.31e-04 -11.0 3.16e-03 - 1.00e+00 1.00e+00h 1
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
10 1.2085150e-01 2.91e-11 1.16e-10 -11.0 1.36e-05 - 1.00e+00 1.00e+00h 1
Number of Iterations....: 10
(scaled) (unscaled)
Objective...............: 1.2085149535785018e-01 1.2085149535785018e-01
Dual infeasibility......: 1.1646050220519939e-10 1.1646050220519939e-10
Constraint violation....: 2.9143798485620209e-11 2.9143798485620209e-11
Complementarity.........: 0.0000000000000000e+00 0.0000000000000000e+00
Overall NLP error.......: 1.1646050220519939e-10 1.1646050220519939e-10
Number of objective function evaluations = 11
Number of objective gradient evaluations = 11
Number of equality constraint evaluations = 11
Number of inequality constraint evaluations = 0
Number of equality constraint Jacobian evaluations = 11
Number of inequality constraint Jacobian evaluations = 0
Number of Lagrangian Hessian evaluations = 0
Total CPU secs in IPOPT (w/o function evaluations) = 1.744
Total CPU secs in NLP function evaluations = 0.017
EXIT: Optimal Solution Found.
=====================================
Known value of p = 10.0
Identified value of p = 9.99895472407
=====================================
This is Ipopt version 3.11, running with linear solver ma27.
Number of nonzeros in equality constraint Jacobian...: 49990
Number of nonzeros in inequality constraint Jacobian.: 0
Number of nonzeros in Lagrangian Hessian.............: 0
Total number of variables............................: 10001
variables with only lower bounds: 0
variables with lower and upper bounds: 0
variables with only upper bounds: 0
Total number of equality constraints.................: 9998
Total number of inequality constraints...............: 0
inequality constraints with only lower bounds: 0
inequality constraints with lower and upper bounds: 0
inequality constraints with only upper bounds: 0
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
0 0.0000000e+00 5.59e+01 0.00e+00 0.0 0.00e+00 - 0.00e+00 0.00e+00 0
1 1.2875001e-01 9.44e-02 6.05e-01 -11.0 5.00e-01 - 1.00e+00 1.00e+00h 1
2 1.2866631e-01 2.30e-05 1.77e-04 -11.0 1.12e-02 - 1.00e+00 1.00e+00h 1
3 1.2866424e-01 1.25e-07 1.07e-04 -11.0 1.09e-03 - 1.00e+00 1.00e+00h 1
4 1.2866402e-01 9.30e-10 1.20e-04 -11.0 9.60e-05 - 1.00e+00 1.00e+00h 1
5 1.2866157e-01 6.50e-08 1.08e-04 -11.0 7.69e-04 - 1.00e+00 1.00e+00h 1
6 1.2865807e-01 1.31e-07 8.92e-05 -11.0 1.03e-03 - 1.00e+00 1.00e+00h 1
7 1.2865214e-01 4.43e-07 5.89e-05 -11.0 1.83e-03 - 1.00e+00 1.00e+00h 1
8 1.2864643e-01 4.46e-07 1.01e-05 -11.0 1.81e-03 - 1.00e+00 1.00e+00h 1
9 1.2864643e-01 1.56e-11 3.41e-10 -11.0 1.05e-05 - 1.00e+00 1.00e+00h 1
Number of Iterations....: 9
(scaled) (unscaled)
Objective...............: 1.2864643016495872e-01 1.2864643016495872e-01
Dual infeasibility......: 3.4081415345595632e-10 3.4081415345595632e-10
Constraint violation....: 1.5624834759364603e-11 1.5624834759364603e-11
Complementarity.........: 0.0000000000000000e+00 0.0000000000000000e+00
Overall NLP error.......: 3.4081415345595632e-10 3.4081415345595632e-10
Number of objective function evaluations = 10
Number of objective gradient evaluations = 10
Number of equality constraint evaluations = 10
Number of inequality constraint evaluations = 0
Number of equality constraint Jacobian evaluations = 10
Number of inequality constraint Jacobian evaluations = 0
Number of Lagrangian Hessian evaluations = 0
Total CPU secs in IPOPT (w/o function evaluations) = 0.374
Total CPU secs in NLP function evaluations = 0.025
EXIT: Optimal Solution Found.
=====================================
Known value of p = 10.0
Identified value of p = 10.0047361779
=====================================
This is Ipopt version 3.11, running with linear solver ma57.
Number of nonzeros in equality constraint Jacobian...: 49990
Number of nonzeros in inequality constraint Jacobian.: 0
Number of nonzeros in Lagrangian Hessian.............: 0
Total number of variables............................: 10001
variables with only lower bounds: 0
variables with lower and upper bounds: 0
variables with only upper bounds: 0
Total number of equality constraints.................: 9998
Total number of inequality constraints...............: 0
inequality constraints with only lower bounds: 0
inequality constraints with lower and upper bounds: 0
inequality constraints with only upper bounds: 0
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
0 0.0000000e+00 6.28e+01 0.00e+00 0.0 0.00e+00 - 0.00e+00 0.00e+00 0
1 1.7614998e-01 2.99e+00 2.86e+01 -11.0 2.13e+01 - 1.00e+00 1.00e+00h 1
2 1.5671462e-01 9.42e-02 7.11e+00 -11.0 1.34e+00 - 1.00e+00 1.00e+00h 1
3 1.5525607e-01 2.28e-04 2.36e-01 -11.0 4.36e-02 - 1.00e+00 1.00e+00h 1
4 1.5513574e-01 9.61e-09 2.82e-04 -11.0 2.82e-04 - 1.00e+00 1.00e+00h 1
5 1.2410780e-01 1.71e-03 3.73e-04 -11.0 1.11e-01 - 1.00e+00 1.00e+00f 1
6 1.2410592e-01 9.32e-07 8.07e-06 -11.0 2.21e-03 - 1.00e+00 1.00e+00h 1
7 1.2410594e-01 5.44e-11 9.66e-10 -11.0 1.64e-05 - 1.00e+00 1.00e+00h 1
Number of Iterations....: 7
(scaled) (unscaled)
Objective...............: 1.2410593595776956e-01 1.2410593595776956e-01
Dual infeasibility......: 9.6593344864122314e-10 9.6593344864122314e-10
Constraint violation....: 5.4365401069844665e-11 5.4365401069844665e-11
Complementarity.........: 0.0000000000000000e+00 0.0000000000000000e+00
Overall NLP error.......: 9.6593344864122314e-10 9.6593344864122314e-10
Number of objective function evaluations = 8
Number of objective gradient evaluations = 8
Number of equality constraint evaluations = 8
Number of inequality constraint evaluations = 0
Number of equality constraint Jacobian evaluations = 8
Number of inequality constraint Jacobian evaluations = 0
Number of Lagrangian Hessian evaluations = 0
Total CPU secs in IPOPT (w/o function evaluations) = 0.266
Total CPU secs in NLP function evaluations = 0.020
EXIT: Optimal Solution Found.
=====================================
Known value of p = 10.0
Identified value of p = 10.0005792456
=====================================
This is Ipopt version 3.11, running with linear solver ma77.
Number of nonzeros in equality constraint Jacobian...: 49990
Number of nonzeros in inequality constraint Jacobian.: 0
Number of nonzeros in Lagrangian Hessian.............: 0
Total number of variables............................: 10001
variables with only lower bounds: 0
variables with lower and upper bounds: 0
variables with only upper bounds: 0
Total number of equality constraints.................: 9998
Total number of inequality constraints...............: 0
inequality constraints with only lower bounds: 0
inequality constraints with lower and upper bounds: 0
inequality constraints with only upper bounds: 0
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
0 0.0000000e+00 6.21e+01 0.00e+00 0.0 0.00e+00 - 0.00e+00 0.00e+00 0
1 1.4065890e-01 1.74e+00 1.11e+01 -11.0 1.32e+01 - 1.00e+00 1.00e+00h 1
2 1.3534441e-01 1.51e-02 7.70e+00 -11.0 3.18e-01 - 1.00e+00 1.00e+00h 1
3 1.3474808e-01 5.60e-05 1.82e-01 -11.0 1.72e-02 - 1.00e+00 1.00e+00h 1
4 1.3178245e-01 2.48e-03 3.24e-03 -11.0 1.41e-01 - 1.00e+00 1.00e+00h 1
5 1.2114347e-01 6.02e-04 3.73e-03 -11.0 6.57e-02 - 1.00e+00 1.00e+00h 1
6 1.2112398e-01 2.01e-07 3.00e-03 -11.0 8.77e-04 - 1.00e+00 1.00e+00h 1
7 1.2101840e-01 8.57e-06 7.63e-04 -11.0 8.49e-03 - 1.00e+00 1.00e+00h 1
8 1.2101835e-01 3.04e-09 5.07e-06 -11.0 1.62e-04 - 1.00e+00 1.00e+00h 1
9 1.2101835e-01 1.75e-13 5.52e-11 -11.0 1.27e-06 - 1.00e+00 1.00e+00h 1
Number of Iterations....: 9
(scaled) (unscaled)
Objective...............: 1.2101834869543603e-01 1.2101834869543603e-01
Dual infeasibility......: 5.5174831066831520e-11 5.5174831066831520e-11
Constraint violation....: 1.7497114868092467e-13 1.7497114868092467e-13
Complementarity.........: 0.0000000000000000e+00 0.0000000000000000e+00
Overall NLP error.......: 5.5174831066831520e-11 5.5174831066831520e-11
Number of objective function evaluations = 10
Number of objective gradient evaluations = 10
Number of equality constraint evaluations = 10
Number of inequality constraint evaluations = 0
Number of equality constraint Jacobian evaluations = 10
Number of inequality constraint Jacobian evaluations = 0
Number of Lagrangian Hessian evaluations = 0
Total CPU secs in IPOPT (w/o function evaluations) = 1.813
Total CPU secs in NLP function evaluations = 0.020
EXIT: Optimal Solution Found.
=====================================
Known value of p = 10.0
Identified value of p = 10.0009200811
=====================================
This is Ipopt version 3.11, running with linear solver ma86.
Number of nonzeros in equality constraint Jacobian...: 49990
Number of nonzeros in inequality constraint Jacobian.: 0
Number of nonzeros in Lagrangian Hessian.............: 0
Total number of variables............................: 10001
variables with only lower bounds: 0
variables with lower and upper bounds: 0
variables with only upper bounds: 0
Total number of equality constraints.................: 9998
Total number of inequality constraints...............: 0
inequality constraints with only lower bounds: 0
inequality constraints with lower and upper bounds: 0
inequality constraints with only upper bounds: 0
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
0 0.0000000e+00 5.79e+01 0.00e+00 0.0 0.00e+00 - 0.00e+00 0.00e+00 0
1 1.2454337e-01 4.67e-01 3.66e+00 -11.0 3.30e+00 - 1.00e+00 1.00e+00h 1
2 1.2077429e-01 1.52e-03 1.29e+00 -11.0 8.64e-02 - 1.00e+00 1.00e+00h 1
3 1.2074944e-01 1.01e-06 7.01e-03 -11.0 3.09e-03 - 1.00e+00 1.00e+00h 1
4 1.2070615e-01 6.95e-06 1.11e-02 -11.0 7.25e-03 - 1.00e+00 1.00e+00h 1
5 1.2070311e-01 5.43e-08 8.12e-03 -11.0 5.86e-04 - 1.00e+00 1.00e+00h 1
6 1.2069966e-01 2.28e-07 1.17e-03 -11.0 1.26e-03 - 1.00e+00 1.00e+00h 1
7 1.2069964e-01 1.67e-09 1.31e-04 -11.0 1.16e-04 - 1.00e+00 1.00e+00h 1
8 1.2069964e-01 2.67e-11 1.78e-10 -11.0 1.42e-05 - 1.00e+00 1.00e+00h 1
Number of Iterations....: 8
(scaled) (unscaled)
Objective...............: 1.2069964078901724e-01 1.2069964078901724e-01
Dual infeasibility......: 1.7847271035174861e-10 1.7847271035174861e-10
Constraint violation....: 2.6659563445718959e-11 2.6659563445718959e-11
Complementarity.........: 0.0000000000000000e+00 0.0000000000000000e+00
Overall NLP error.......: 1.7847271035174861e-10 1.7847271035174861e-10
Number of objective function evaluations = 9
Number of objective gradient evaluations = 9
Number of equality constraint evaluations = 9
Number of inequality constraint evaluations = 0
Number of equality constraint Jacobian evaluations = 9
Number of inequality constraint Jacobian evaluations = 0
Number of Lagrangian Hessian evaluations = 0
Total CPU secs in IPOPT (w/o function evaluations) = 1.052
Total CPU secs in NLP function evaluations = 0.013
EXIT: Optimal Solution Found.
=====================================
Known value of p = 10.0
Identified value of p = 10.0033343866
=====================================
This is Ipopt version 3.11, running with linear solver ma97.
Number of nonzeros in equality constraint Jacobian...: 49990
Number of nonzeros in inequality constraint Jacobian.: 0
Number of nonzeros in Lagrangian Hessian.............: 0
Total number of variables............................: 10001
variables with only lower bounds: 0
variables with lower and upper bounds: 0
variables with only upper bounds: 0
Total number of equality constraints.................: 9998
Total number of inequality constraints...............: 0
inequality constraints with only lower bounds: 0
inequality constraints with lower and upper bounds: 0
inequality constraints with only upper bounds: 0
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
0 0.0000000e+00 5.85e+01 0.00e+00 0.0 0.00e+00 - 0.00e+00 0.00e+00 0
1 2.3397713e-01 2.76e+00 4.69e+01 -11.0 1.62e+01 - 1.00e+00 1.00e+00h 1
2 1.5233026e-01 1.28e-01 7.14e+01 -11.0 9.93e-01 - 1.00e+00 1.00e+00h 1
3 1.5199146e-01 2.45e-04 2.10e+00 -11.0 5.52e-02 - 1.00e+00 1.00e+00h 1
WARNING: Problem in step computation; switching to emergency mode.
4r 1.5199146e-01 2.45e-04 9.99e+02 -3.6 0.00e+00 20.0 0.00e+00 0.00e+00R 1
5r 1.5199043e-01 6.17e-05 7.91e-01 -9.0 2.45e-01 - 1.00e+00 9.90e-04f 1
WARNING: Problem in step computation; switching to emergency mode.
6r 1.5199043e-01 6.17e-05 9.99e+02 -4.2 0.00e+00 20.0 0.00e+00 0.00e+00R 1
7r 1.5198918e-01 1.56e-05 6.62e-01 -9.0 6.18e-02 - 1.00e+00 9.90e-04f 1
WARNING: Problem in step computation; switching to emergency mode.
8r 1.5198918e-01 1.56e-05 9.99e+02 -4.8 0.00e+00 20.0 0.00e+00 0.00e+00R 1
9r 1.5198879e-01 3.93e-06 2.12e-02 -9.0 2.31e-02 - 1.00e+00 9.90e-04f 1
WARNING: Problem in step computation; switching to emergency mode.
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
10r 1.5198879e-01 3.93e-06 9.99e+02 -5.4 0.00e+00 20.0 0.00e+00 0.00e+00R 1
11r 1.5198870e-01 9.90e-07 6.69e-03 -9.0 7.12e-03 - 1.00e+00 9.90e-04f 1
WARNING: Problem in step computation; switching to emergency mode.
12r 1.5198870e-01 9.90e-07 9.99e+02 -6.0 0.00e+00 20.0 0.00e+00 0.00e+00R 1
13r 1.5198868e-01 2.50e-07 1.11e-03 -9.0 1.19e-03 - 1.00e+00 9.90e-04f 1
14 1.2593052e-01 1.49e-03 1.61e-03 -11.0 1.03e-01 - 1.00e+00 1.00e+00f 1
15 1.2590571e-01 6.25e-06 6.87e-05 -11.0 5.39e-03 - 1.00e+00 1.00e+00h 1
16 1.2590586e-01 1.23e-09 1.42e-09 -11.0 8.66e-05 - 1.00e+00 1.00e+00h 1
Number of Iterations....: 16
(scaled) (unscaled)
Objective...............: 1.2590586317570238e-01 1.2590586317570238e-01
Dual infeasibility......: 1.4242203697856573e-09 1.4242203697856573e-09
Constraint violation....: 1.2284639971937850e-09 1.2284639971937850e-09
Complementarity.........: 0.0000000000000000e+00 0.0000000000000000e+00
Overall NLP error.......: 1.4242203697856573e-09 1.4242203697856573e-09
Number of objective function evaluations = 17
Number of objective gradient evaluations = 12
Number of equality constraint evaluations = 17
Number of inequality constraint evaluations = 0
Number of equality constraint Jacobian evaluations = 17
Number of inequality constraint Jacobian evaluations = 0
Number of Lagrangian Hessian evaluations = 0
Total CPU secs in IPOPT (w/o function evaluations) = 5.042
Total CPU secs in NLP function evaluations = 0.030
EXIT: Optimal Solution Found.
=====================================
Known value of p = 10.0
Identified value of p = 10.0013889246
=====================================
Very nice, MA57 was clearly much faster than the others.
Can that conclusion be generalized to other and larger problems?
If so, I should probably ask Anne to try again to compile the Matlab interface with MA57. I think she is using the version with MUMPS that they distributed as binary. I did it once a few years ago, but there were some frustrations before I finally got it working.
Anne, click on the #13 link if you want to see Jason's test results.
Ton
On 12/4/2014 3:03 PM, Jason K. Moore wrote:
Closed #13 https://github.com/csu-hmc/opty/issues/13.
— Reply to this email directly or view it on GitHub https://github.com/csu-hmc/opty/issues/13#event-202872967.
I think MA57 is the one you should start with. From this and what I've read it is the better solver. I still haven't tried pardiso or wsmp. I've had trouble getting pardiso to link properly during compiling.
I believe MA57 may come with the Matlab binary. I think that Intel MKL includes MA57 and Matlab links to the MKL, and thus MA57 is available for IPOPT to use. So there may be no recompiling necessary for it.
If you want me to run this for bigger problems I can. It's simple to do now.
I tried using the compiled version that is included, but it does not give reproduceable results... So I will have to look into if I can compile it such that it will be reproduceable.
2014-12-04 15:20 GMT-05:00 Jason K. Moore notifications@github.com:
I believe MA57 may come with the Matlab binary. I think that Intel MKL includes MA57 and Matlab links to the MKL, and thus MA57 is available for IPOPT to use. So there may be no recompiling necessary for it.
— Reply to this email directly or view it on GitHub https://github.com/csu-hmc/opty/issues/13#issuecomment-65696732.
I'm not sure which of the sparse solvers give best performance. Ideally if any can be parallelized, that would help.