custom installation is ignored, still running the default version

[goghino]

I have compiled my own Ipopt shared library using the latest version (3.13.3) with Pardiso. I have added it to Julia following the custom installation steps:

ENV["JULIA_IPOPT_LIBRARY_PATH"] = "/home/kardos/PowerModels-benchmarks/ipopt-git/myinstall/lib"
ENV["JULIA_IPOPT_EXECUTABLE_PATH"] = "/home/kardos/PowerModels-benchmarks/ipopt-git/myinstall/bin"
import Pkg; Pkg.build("Ipopt")

The installation does not produce any errors, when calling the BinaryProvider directly, this is the output:

JULIA_IPOPT_LIBRARY_PATH="/home/kardos/PowerModels-benchmarks/ipopt-git/myinstall/lib" JULIA_IPOPT_EXECUTABLE_PATH="/home/kardos/PowerModels-benchmarks/ipopt-git/myinstall/bin"  julia /home/kardos/.julia/packages/Ipopt/YJcu4/deps/build.jl --verbose
[ Info: Found a valid dl path libipopt.so while looking for libipopt
[ Info: /home/kardos/PowerModels-benchmarks/ipopt-git/myinstall/lib/libipopt.so matches our search criteria of libipopt
[ Info: Found a valid dl path libipopt.so while looking for libipopt
[ Info: /home/kardos/PowerModels-benchmarks/ipopt-git/myinstall/lib/libipopt.so matches our search criteria of libipopt
[ Info: Found a valid dl path libipopt.so while looking for libipopt
[ Info: /home/kardos/PowerModels-benchmarks/ipopt-git/myinstall/lib/libipopt.so matches our search criteria of libipopt

However, Julia is still using the default Ipopt 3.13.2, running with linear solver mumps. I am using fresh installation of Julia Version 1.4.2 (2020-05-23). How can I force Julia to use my Ipopt binary, instead of the one that is used by default?

[odow]

This should have worked. What is the output of

/home/kardos/.julia/packages/Ipopt/YJcu4/deps/deps.jl

[goghino]

$ cat /home/kardos/.julia/packages/Ipopt/YJcu4/deps/deps.jl
## This file autogenerated by BinaryProvider.write_deps_file().
## Do not edit.
##
## Include this file within your main top-level source, and call
## `check_deps()` from within your module's `__init__()` method

if isdefined((@static VERSION < v"0.7.0-DEV.484" ? current_module() : @__MODULE__), :Compat)
    import Compat.Libdl
elseif VERSION >= v"0.7.0-DEV.3382"
    import Libdl
end
const libipopt = joinpath(dirname(@__FILE__), "../../../../../PowerModels-benchmarks/ipopt-git/myinstall/lib/libipopt.so")
const amplexe = joinpath(dirname(@__FILE__), "../../../../../PowerModels-benchmarks/ipopt-git/myinstall/bin/ipopt")
function check_deps()
    global libipopt
    if !isfile(libipopt)
        error("$(libipopt) does not exist, Please re-run Pkg.build(\"Ipopt\"), and restart Julia.")
    end

    if Libdl.dlopen_e(libipopt) in (C_NULL, nothing)
        error("$(libipopt) cannot be opened, Please re-run Pkg.build(\"Ipopt\"), and restart Julia.")
    end

    global amplexe
    if !isfile(amplexe)
        error("$(amplexe) does not exist, Please re-run Pkg.build(\"Ipopt\"), and restart Julia.")
    end

    libpaths = split(get(ENV, "LD_LIBRARY_PATH", ""), ":")
    if !("/cluster_nfs/Data_Apps/home/kardos/privateapps/julia/julia-1.4.2/bin/../lib/julia" in libpaths)
        push!(libpaths, "/cluster_nfs/Data_Apps/home/kardos/privateapps/julia/julia-1.4.2/bin/../lib/julia")
    end
    ENV["LD_LIBRARY_PATH"] = join(filter(!isempty, libpaths), ":")

end

[odow]

Looks like that worked. Did you close Julia and open a new session?

In a new Julia session, what is the output of

using Ipopt
Ipopt.libipopt

[goghino]

Yes, I closed Julia and started a new session. Attaching the output for the demo example (I think it is hs071)

julia> using Ipopt
julia> Ipopt.libipopt
"libipopt.so.3"

$ julia customIpopt.jl

******************************************************************************
This program contains Ipopt, a library for large-scale nonlinear optimization.
 Ipopt is released as open source code under the Eclipse Public License (EPL).
         For more information visit http://projects.coin-or.org/Ipopt
******************************************************************************

This is Ipopt version 3.13.2, running with linear solver mumps.
NOTE: Other linear solvers might be more efficient (see Ipopt documentation).

Number of nonzeros in equality constraint Jacobian...:        4
Number of nonzeros in inequality constraint Jacobian.:        4
Number of nonzeros in Lagrangian Hessian.............:       10

Total number of variables............................:        4
                     variables with only lower bounds:        0
                variables with lower and upper bounds:        4
                     variables with only upper bounds:        0
Total number of equality constraints.................:        1
Total number of inequality constraints...............:        1
        inequality constraints with only lower bounds:        1
   inequality constraints with lower and upper bounds:        0
        inequality constraints with only upper bounds:        0

iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls
   0  1.6109693e+01 1.12e+01 5.28e-01  -1.0 0.00e+00    -  0.00e+00 0.00e+00   0
   1  1.6982239e+01 7.30e-01 1.02e+01  -1.0 6.11e-01    -  7.19e-02 1.00e+00f  1
   2  1.7318411e+01 3.60e-02 5.05e-01  -1.0 1.61e-01    -  1.00e+00 1.00e+00h  1
   3  1.6849424e+01 2.78e-01 6.68e-02  -1.7 2.85e-01    -  7.94e-01 1.00e+00h  1
   4  1.7051199e+01 4.71e-03 2.78e-03  -1.7 6.06e-02    -  1.00e+00 1.00e+00h  1
   5  1.7011979e+01 7.19e-03 8.50e-03  -3.8 3.66e-02    -  9.45e-01 9.98e-01h  1
   6  1.7014271e+01 1.74e-05 9.78e-06  -3.8 3.33e-03    -  1.00e+00 1.00e+00h  1
   7  1.7014021e+01 1.23e-07 1.82e-07  -5.7 2.69e-04    -  1.00e+00 1.00e+00h  1
   8  1.7014017e+01 1.77e-11 2.52e-11  -8.6 3.32e-06    -  1.00e+00 1.00e+00h  1

Number of Iterations....: 8

                                   (scaled)                 (unscaled)
Objective...............:   1.7014017145179157e+01    1.7014017145179157e+01
Dual infeasibility......:   2.5162491973736561e-11    2.5162491973736561e-11
Constraint violation....:   1.7720935829856899e-11    1.7720935829856899e-11
Complementarity.........:   2.5277100427932987e-09    2.5277100427932987e-09
Overall NLP error.......:   2.5277100427932987e-09    2.5277100427932987e-09

Number of objective function evaluations             = 9
Number of objective gradient evaluations             = 9
Number of equality constraint evaluations            = 9
Number of inequality constraint evaluations          = 9
Number of equality constraint Jacobian evaluations   = 9
Number of inequality constraint Jacobian evaluations = 9
Number of Lagrangian Hessian evaluations             = 8
Total CPU secs in IPOPT (w/o function evaluations)   =      0.083
Total CPU secs in NLP function evaluations           =      0.014

EXIT: Optimal Solution Found.
Solve_Succeeded
[1.0, 4.7429996418092975, 3.8211499817883072, 1.3794082897556978]
17.014017145179157

[odow]

It looks like it is loading a different version of Ipopt.jl for some reason.

Does pathof(Ipopt) match the one we tried before? If not, try rebuilding.

[goghino]

I have already tried rebuilding, I have tried setting the JULIAIPOPT* env variables before I call Pkg.add("Ipopt"), still the default Ipopt is called

julia> pathof(Ipopt)
"/home/kardos/.julia/packages/Ipopt/YJcu4/src/Ipopt.jl"

[odow]

Are the environment variables set in your new session? Make sure they are set before you run using Ipopt. https://github.com/jump-dev/Ipopt.jl/blob/291436532375db9154aec6a562c3781f65ff4967/src/Ipopt.jl#L5-L7

[goghino]

I have removed my $/.julia folder, reinstalled Ipopt and it is working now. If I simply removed Ipopt from the package manager and reinstalled it did not help. Thanks for your help, Oscar.

kardos@icsmaster01 (master):~/PowerModels-benchmarks$ julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.4.2 (2020-05-23)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> ENV["JULIA_IPOPT_LIBRARY_PATH"] = "/home/kardos/PowerModels-benchmarks/ipopt-git/myinstall/lib"
"/home/kardos/PowerModels-benchmarks/ipopt-git/myinstall/lib"

julia> ENV["JULIA_IPOPT_EXECUTABLE_PATH"] = "/home/kardos/PowerModels-benchmarks/ipopt-git/myinstall/bin"
"/home/kardos/PowerModels-benchmarks/ipopt-git/myinstall/bin"

julia> import Pkg

julia> Pkg.add("Ipopt")
   Updating registry at `~/.julia/registries/General`
   Updating git-repo `https://github.com/JuliaRegistries/General.git`
  Resolving package versions...
  [...]
   Building Ipopt → `~/.julia/packages/Ipopt/YJcu4/deps/build.log`

julia> exit()
kardos@icsmaster01 (master):~/PowerModels-benchmarks$ julia customIpopt.jl

******************************************************************************
This program contains Ipopt, a library for large-scale nonlinear optimization.
 Ipopt is released as open source code under the Eclipse Public License (EPL).
         For more information visit https://github.com/coin-or/Ipopt
******************************************************************************
This is Ipopt version 3.13.3, running with linear solver pardiso.

[odow]

Weird. I don't know what the issue was. Good you got it sorted.