How to get HSL solvers working on Windows?

[emmanueljordy]

Hi Everyone,

I have been using cyipopt for some time now with the mumps solver, which proved to not be optimal for my task so I decided to try different solvers.

I first updated Ipopt to the latest release 3.13.2, by downloading the file Ipopt-3.13.2-win64-msvs2019-md.zip, dumping its content into the cyipopt directory, and updating setup.py. cyipopt works well with the new default pardiso solver.

Next, I obtained HSL files from http://www.hsl.rl.ac.uk/ipopt/, and I was able to build the library libhsl.dll and libmetis.dll. Everything went well during the build.

Could you please tell me how I can make cyipopt detect the hsl solvers? wenever I try to use the option 'linear_solver':'ma27', or anyother from HSL it results in an error.

Has someone been able to use HSL solvers with cyipopt using the LinearSolverLoader?

[moorepants]

Thanks for the question. Please post the code you tried and the resulting error messages.

[emmanueljordy]

Thanks for the answer. The code is part of an optimization problem where I use the minimize_ipopt function.

rdcreg_optim = minimize_ipopt(optim_func,v0,args = optim_args,constraints = ball_const,options = {'max_iter':1000,'linear_solver':'ma27'})

Next I print rdcreg_optim and get :

  fun: 0.0
    info: {'status_msg': b'Invalid option encountered.', 'obj_val': 0.0, 'x': array([1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
       1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
       1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.]), 'mult_g': array([0., 0., 0., 0.]), 'mult_x_L': array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
       0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
       0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]), 'g': array([0., 0., 0., 0.]), 'mult_x_U': array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
       0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
       0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]), 'status': -12}
 message: b'Invalid option encountered.'
    nfev: 0
     nit: 0
    njev: 0
  status: -12
 success: False
       x: array([1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
       1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
       1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.])

it's saying invalid option encountered. but ma27 is a correct option for linear_solver.

[moorepants]

One note, is that cyipopt does not support ipopt >= 3.13 yet. It may work, but there was major restructuring of ipopt in version 3.13. I recommend using the version 3.11 listed in the documentation for windows in the README.

[moorepants]

Can you try passing in byte strings: b'linear_solver': b'ma27'?

[emmanueljordy]

I have tried with byte strings : b'linear_solver': b'ma27' and it still says the same. Note that I can choose between mumps and pardiso, meaning that if I do either of: 'linear_solver' : 'mumps' 'linear_solver' : 'pardiso'

Everything works fine. So I think it's mostly about pointing cyipopt to the libhsl.dll. I have tried adding the path to libhsl.dll to the PATH environment variable, but it still did not solve the problem.

I have also tried using version 3.11, but still have the same problem.

[moorepants]

I think this may be a question more for the ipopt forums. It sounds like your ipopt installation is unlikely linking to these libraries. Try running ipopt as a compiled C/C++ program and using those libraries. You may get the same error. If so, you'll need to reach out to the ipopt support channels for help.

[donboyd5]

@emmanueljordy, would you be willing to post information (e.g., script) on how you compiled the HSL solvers as a Windows dll?

I am eager to do this because Mumps cannot handle some of the problems I have, but I have not yet figured out how to do this. I have been in touch with the folks at HSL but the dll they were able to provide did not work with my cyipopt installation. Their own efforts at creating a Windows dll encountered some difficulty and I don't think they have time to focus attention on this.

Also, if I can get the HSL folks to provide you with the ok, would you be willing to provide me with a 64-bit Windows dll for HSL? I suspect they would value having a working dll that they can provide to others, also, if you are willing to provide it to them.

Many thanks for considering both questions.

[ashander]

I suspect it's related to to version of IPOPT not being built with the "dynamic library loader"

From the hsl readme:

Installation

Drop the .dll files into the directory containing your program, or into another directory on your PATH. If Ipopt has been built with the dynamic library loader enabled, everything should work.

[patrickocal]

cyipopt does not support ipopt >= 3.13

Is this still true?

[moorepants]

We are testing with the conda forge ipopt binaries as per:

        os: [ubuntu-latest, macos-latest, windows-latest]
        python-version: [3.6, 3.7, 3.8, 3.9]
        ipopt-version: [3.12, 3.13, 3.14]

So it works with that combination of os, python, and ipopt. But I don't know the state of working with the official ipopt binaries from coinor.

[moorepants]

We have documentation of using the HSL solvers in the online documentation, so closing this.

[avdudchenko]

Hello, adding a new comment to this.

The current instructions for HSL use are for Linux, but not for windows. These instructions do not seem to apply to windows, -e.g. the steps to find libipopt.so do not apply. Can some one give me advice on how to go about getting HSL solvers to work with conda installed binaries on windows? Thank you.

Update: For anyone trying to do it - download the binary version of HSL archive Copy all ".dll" files from "bin" folder in downloaded zip into "yourcondalocation/envs/env-name/Library/bin" folder. This should then allow you to import HSL solvers (e.g. MA27 etc.)

This seems a lot simpler then current instructions, I would request that some one else verify this approach and add instructions to docs?

Thanks!

[moorepants]

This seems a lot simpler then current instructions, I would request that some one else verify this approach and add instructions to docs?

You could be that person :)

[avdudchenko]

This seems a lot simpler then current instructions, I would request that some one else verify this approach and add instructions to docs?

You could be that person :) Add PR: https://github.com/mechmotum/cyipopt/pull/254