Crash on Windows when optimizing with SPRAL_jll

[niclasmattsson]

Not sure where the problem is so I hope it's OK if I open it here. Here's a quick test of the diet problem (from JuMP docs), first solved using Ipopt/MUMPS and then an attempt with SPRAL.

I'm not 100% sure I set everything up correctly though. I set the environment variables OMP_* as noted in the Ipopt README, started Julia and entered the diet problem with JuMP, then ]add SPRAL_jll and import SPRAL_jll, then finally what you see below. Did I miss anything obvious?

julia> optimize!(model)

******************************************************************************
This program contains Ipopt, a library for large-scale nonlinear optimization.
 Ipopt is released as open source code under the Eclipse Public License (EPL).
         For more information visit https://github.com/coin-or/Ipopt
******************************************************************************

This is Ipopt version 3.14.13, running with linear solver MUMPS 5.6.0.

Number of nonzeros in equality constraint Jacobian...:        0
Number of nonzeros in inequality constraint Jacobian.:       63
Number of nonzeros in Lagrangian Hessian.............:        0

Total number of variables............................:        9
                     variables with only lower bounds:        9
                variables with lower and upper bounds:        0
                     variables with only upper bounds:        0
Total number of equality constraints.................:        0
Total number of inequality constraints...............:        7
        inequality constraints with only lower bounds:        4
   inequality constraints with lower and upper bounds:        0
        inequality constraints with only upper bounds:        3

iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls
   0  1.7819982e-01 1.77e+03 9.49e-01  -1.0 0.00e+00    -  0.00e+00 0.00e+00   0
   1  2.3371190e-01 1.76e+03 9.44e-01  -1.0 2.91e+02    -  3.56e-03 3.92e-03h  1
   2  2.7286981e-01 1.75e+03 2.87e+00  -1.0 3.14e+02    -  1.18e-02 4.60e-03h  1
   3  3.6655206e-01 1.74e+03 1.04e+00  -1.0 3.08e+02    -  1.96e-03 7.99e-03h  1
   4  5.4477465e-01 1.72e+03 7.28e+00  -1.0 2.99e+02    -  3.48e-02 1.34e-02h  1
   5  5.9527194e-01 1.71e+03 9.45e+00  -1.0 3.07e+02    -  2.64e-02 3.86e-03h  1
   6  7.1663689e-01 1.69e+03 5.91e+01  -1.0 3.04e+02    -  5.95e-02 1.10e-02h  1
   7  1.2179680e+01 0.00e+00 2.50e+00  -1.0 3.03e+02    -  6.80e-02 1.00e+00h  1
   8  1.2178892e+01 0.00e+00 6.10e-01  -1.0 6.34e-01    -  9.92e-01 1.00e+00f  1
   9  1.2144489e+01 0.00e+00 6.80e-01  -1.7 1.83e+00    -  8.58e-01 1.00e+00f  1
iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls
  10  1.1930689e+01 0.00e+00 1.25e+00  -1.7 1.43e+01    -  1.00e+00 5.25e-01f  1
  11  1.1955580e+01 0.00e+00 2.00e-07  -1.7 9.77e-01    -  1.00e+00 1.00e+00f  1
  12  1.1879277e+01 0.00e+00 1.46e-01  -2.5 1.21e+00    -  7.86e-01 1.00e+00f  1
  13  1.1853187e+01 0.00e+00 2.83e-08  -2.5 1.38e+00    -  1.00e+00 1.00e+00f  1
  14  1.1830327e+01 0.00e+00 5.69e-04  -3.8 7.94e-01    -  9.31e-01 1.00e+00f  1
  15  1.1828872e+01 0.00e+00 6.50e-07  -5.7 5.56e-02    -  1.00e+00 1.00e+00f  1
  16  1.1828861e+01 0.00e+00 2.58e-14  -8.6 1.73e-04    -  1.00e+00 1.00e+00f  1

Number of Iterations....: 16

                                   (scaled)                 (unscaled)
Objective...............:   1.1828860918557460e+01    1.1828860918557460e+01
Dual infeasibility......:   2.5757174171303632e-14    2.5757174171303632e-14
Constraint violation....:   0.0000000000000000e+00    0.0000000000000000e+00
Variable bound violation:   8.1538046250142944e-09    8.1538046250142944e-09
Complementarity.........:   2.7287960622893697e-09    2.7287960622893697e-09
Overall NLP error.......:   2.7287960622893697e-09    2.7287960622893697e-09

Number of objective function evaluations             = 17
Number of objective gradient evaluations             = 17
Number of equality constraint evaluations            = 0
Number of inequality constraint evaluations          = 17
Number of equality constraint Jacobian evaluations   = 0
Number of inequality constraint Jacobian evaluations = 1
Number of Lagrangian Hessian evaluations             = 1
Total seconds in IPOPT                               = 0.060

EXIT: Optimal Solution Found.

julia> set_attribute(model, "linear_solver", "spral")

julia> optimize!(model)
This is Ipopt version 3.14.13, running with linear solver spral.

Number of nonzeros in equality constraint Jacobian...:        0
Number of nonzeros in inequality constraint Jacobian.:       63
Number of nonzeros in Lagrangian Hessian.............:        0

[process exited with code 3221226356 (0xc0000374)]

[odow]

What is versioninfo() and import Pkg; Pkg.pkg"st -m"?

Does it work if you run Julia in Windows Subsystem for Linux?

[niclasmattsson]

Thanks and sorry for my not having time to revisit this until today.

julia> versioninfo()
Julia Version 1.9.1
Commit 147bdf428c (2023-06-07 08:27 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: 16 × AMD Ryzen 9 6900HX with Radeon Graphics
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-14.0.6 (ORCJIT, znver3)
  Threads: 14 on 16 virtual cores
Environment:
  JULIA_EDITOR = "C:\Users\niclas\AppData\Local\Programs\Microsoft VS Code\Code.exe"
  JULIA_NUM_THREADS = 14
  JULIA_PKG_DEVDIR = C:\Stuff\Julia

(testENV) pkg> st -m
Status `C:\Stuff\Julia\testENV\Manifest.toml`
  [6e4b80f9] BenchmarkTools v1.3.2
  [523fee87] CodecBzip2 v0.7.2
  [944b1d66] CodecZlib v0.7.1
  [bbf7d656] CommonSubexpressions v0.3.0
  [34da2185] Compat v4.6.1
  [a8cc5b0e] Crayons v4.1.1
  [9a962f9c] DataAPI v1.15.0
  [a93c6f00] DataFrames v1.5.0
  [864edb3b] DataStructures v0.18.13
  [e2d170a0] DataValueInterfaces v1.0.0
  [163ba53b] DiffResults v1.1.0
  [b552c78f] DiffRules v1.15.1
  [ffbed154] DocStringExtensions v0.9.3
  [59287772] Formatting v0.4.2
  [f6369f11] ForwardDiff v0.10.35
  [842dd82b] InlineStrings v1.4.0
  [41ab1584] InvertedIndices v1.3.0
  [b6b21f68] Ipopt v1.4.1
  [92d709cd] IrrationalConstants v0.2.2
  [82899510] IteratorInterfaceExtensions v1.0.0
  [692b3bcd] JLLWrappers v1.4.1
  [682c06a0] JSON v0.21.4
  [4076af6c] JuMP v1.11.1
  [b964fa9f] LaTeXStrings v1.3.0
  [2ab3a3ac] LogExpFunctions v0.3.24
  [1914dd2f] MacroTools v0.5.10
  [b8f27783] MathOptInterface v1.17.1
  [e1d29d7a] Missings v1.1.0
  [d8a4904e] MutableArithmetics v1.3.0
  [77ba4419] NaNMath v1.0.2
  [bac558e1] OrderedCollections v1.6.0
  [69de0a69] Parsers v2.6.0
  [2dfb63ee] PooledArrays v1.4.2
  [aea7be01] PrecompileTools v1.1.2
  [21216c6a] Preferences v1.4.0
  [08abe8d2] PrettyTables v2.2.4
  [189a3867] Reexport v1.2.2
  [91c51154] SentinelArrays v1.4.0
  [66db9d55] SnoopPrecompile v1.0.3
  [a2af1166] SortingAlgorithms v1.1.0
  [276daf66] SpecialFunctions v2.2.0
  [1e83bf80] StaticArraysCore v1.4.0
  [892a3eda] StringManipulation v0.3.0
  [3783bdb8] TableTraits v1.0.1
  [bd369af6] Tables v1.10.1
  [3bb67fe8] TranscodingStreams v0.9.13
  [ae81ac8f] ASL_jll v0.1.3+0
  [6e34b625] Bzip2_jll v1.0.8+0
  [e33a78d0] Hwloc_jll v2.9.1+0
  [9cc047cb] Ipopt_jll v300.1400.1300+0
  [d00139f3] METIS_jll v5.1.2+0
  [d7ed1dd3] MUMPS_seq_jll v500.600.0+0
⌅ [656ef2d0] OpenBLAS32_jll v0.3.21+0
  [efe28fd5] OpenSpecFun_jll v0.5.5+0
  [319450e9] SPRAL_jll v2023.3.29+0
  [0dad84c5] ArgTools v1.1.1
  [56f22d72] Artifacts
  [2a0f44e3] Base64
  [ade2ca70] Dates
  [f43a241f] Downloads v1.6.0
  [7b1f6079] FileWatching
  [9fa8497b] Future
  [b77e0a4c] InteractiveUtils
  [b27032c2] LibCURL v0.6.3
  [76f85450] LibGit2
  [8f399da3] Libdl
  [37e2e46d] LinearAlgebra
  [56ddb016] Logging
  [d6f4376e] Markdown
  [a63ad114] Mmap
  [ca575930] NetworkOptions v1.2.0
  [44cfe95a] Pkg v1.9.0
  [de0858da] Printf
  [9abbd945] Profile
  [3fa0cd96] REPL
  [9a3f8284] Random
  [ea8e919c] SHA v0.7.0
  [9e88b42a] Serialization
  [6462fe0b] Sockets
  [2f01184e] SparseArrays
  [10745b16] Statistics v1.9.0
  [fa267f1f] TOML v1.0.3
  [a4e569a6] Tar v1.10.0
  [8dfed614] Test
  [cf7118a7] UUIDs
  [4ec0a83e] Unicode
  [e66e0078] CompilerSupportLibraries_jll v1.0.2+0
  [deac9b47] LibCURL_jll v7.84.0+0
  [29816b5a] LibSSH2_jll v1.10.2+0
  [c8ffd9c3] MbedTLS_jll v2.28.2+0
  [14a3606d] MozillaCACerts_jll v2022.10.11
  [4536629a] OpenBLAS_jll v0.3.21+4
  [05823500] OpenLibm_jll v0.8.1+0
  [bea87d4a] SuiteSparse_jll v5.10.1+6
  [83775a58] Zlib_jll v1.2.13+0
  [8e850b90] libblastrampoline_jll v5.8.0+0
  [8e850ede] nghttp2_jll v1.48.0+0
  [3f19e933] p7zip_jll v17.4.0+0
Info Packages marked with ⌅ have new versions available but compatibility constraints restrict them from upgrading. To see why use `status --outdated -m`

[niclasmattsson]

New comment to separate output and hopefully avoid confusion. I tried it in WSL as you suggested. It works there, which gives me a nice workaround until this issue is resolved. Here is the same info under WSL:

julia> versioninfo()
Julia Version 1.9.1
Commit 147bdf428cd (2023-06-07 08:27 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 16 × AMD Ryzen 9 6900HX with Radeon Graphics
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-14.0.6 (ORCJIT, znver3)
  Threads: 1 on 16 virtual cores

(testENV) pkg> st -m
Status `~/testENV/Manifest.toml`
  [6e4b80f9] BenchmarkTools v1.3.2
  [523fee87] CodecBzip2 v0.7.2
  [944b1d66] CodecZlib v0.7.1
  [bbf7d656] CommonSubexpressions v0.3.0
  [34da2185] Compat v4.6.1
  [a8cc5b0e] Crayons v4.1.1
  [9a962f9c] DataAPI v1.15.0
  [a93c6f00] DataFrames v1.5.0
  [864edb3b] DataStructures v0.18.13
  [e2d170a0] DataValueInterfaces v1.0.0
  [163ba53b] DiffResults v1.1.0
  [b552c78f] DiffRules v1.15.1
  [ffbed154] DocStringExtensions v0.9.3
  [59287772] Formatting v0.4.2
  [f6369f11] ForwardDiff v0.10.35
  [842dd82b] InlineStrings v1.4.0
  [41ab1584] InvertedIndices v1.3.0
  [b6b21f68] Ipopt v1.4.1
  [92d709cd] IrrationalConstants v0.2.2
  [82899510] IteratorInterfaceExtensions v1.0.0
  [692b3bcd] JLLWrappers v1.4.1
  [682c06a0] JSON v0.21.4
  [4076af6c] JuMP v1.11.1
  [b964fa9f] LaTeXStrings v1.3.0
  [2ab3a3ac] LogExpFunctions v0.3.24
  [1914dd2f] MacroTools v0.5.10
  [b8f27783] MathOptInterface v1.17.1
  [e1d29d7a] Missings v1.1.0
  [d8a4904e] MutableArithmetics v1.3.0
  [77ba4419] NaNMath v1.0.2
  [bac558e1] OrderedCollections v1.6.0
  [69de0a69] Parsers v2.6.0
  [2dfb63ee] PooledArrays v1.4.2
  [aea7be01] PrecompileTools v1.1.2
  [21216c6a] Preferences v1.4.0
  [08abe8d2] PrettyTables v2.2.4
  [189a3867] Reexport v1.2.2
  [91c51154] SentinelArrays v1.4.0
  [66db9d55] SnoopPrecompile v1.0.3
  [a2af1166] SortingAlgorithms v1.1.0
  [276daf66] SpecialFunctions v2.2.0
  [1e83bf80] StaticArraysCore v1.4.0
  [892a3eda] StringManipulation v0.3.0
  [3783bdb8] TableTraits v1.0.1
  [bd369af6] Tables v1.10.1
  [3bb67fe8] TranscodingStreams v0.9.13
  [ae81ac8f] ASL_jll v0.1.3+0
  [6e34b625] Bzip2_jll v1.0.8+0
  [e33a78d0] Hwloc_jll v2.9.1+0
  [9cc047cb] Ipopt_jll v300.1400.1300+0
  [d00139f3] METIS_jll v5.1.2+0
  [d7ed1dd3] MUMPS_seq_jll v500.600.0+0
⌅ [656ef2d0] OpenBLAS32_jll v0.3.21+0
  [efe28fd5] OpenSpecFun_jll v0.5.5+0
  [319450e9] SPRAL_jll v2023.3.29+0
  [0dad84c5] ArgTools v1.1.1
  [56f22d72] Artifacts
  [2a0f44e3] Base64
  [ade2ca70] Dates
  [f43a241f] Downloads v1.6.0
  [7b1f6079] FileWatching
  [9fa8497b] Future
  [b77e0a4c] InteractiveUtils
  [b27032c2] LibCURL v0.6.3
  [76f85450] LibGit2
  [8f399da3] Libdl
  [37e2e46d] LinearAlgebra
  [56ddb016] Logging
  [d6f4376e] Markdown
  [a63ad114] Mmap
  [ca575930] NetworkOptions v1.2.0
  [44cfe95a] Pkg v1.9.0
  [de0858da] Printf
  [9abbd945] Profile
  [3fa0cd96] REPL
  [9a3f8284] Random
  [ea8e919c] SHA v0.7.0
  [9e88b42a] Serialization
  [6462fe0b] Sockets
  [2f01184e] SparseArrays
  [10745b16] Statistics v1.9.0
  [fa267f1f] TOML v1.0.3
  [a4e569a6] Tar v1.10.0
  [8dfed614] Test
  [cf7118a7] UUIDs
  [4ec0a83e] Unicode
  [e66e0078] CompilerSupportLibraries_jll v1.0.2+0
  [deac9b47] LibCURL_jll v7.84.0+0
  [29816b5a] LibSSH2_jll v1.10.2+0
  [c8ffd9c3] MbedTLS_jll v2.28.2+0
  [14a3606d] MozillaCACerts_jll v2022.10.11
  [4536629a] OpenBLAS_jll v0.3.21+4
  [05823500] OpenLibm_jll v0.8.1+0
  [bea87d4a] SuiteSparse_jll v5.10.1+6
  [83775a58] Zlib_jll v1.2.13+0
  [8e850b90] libblastrampoline_jll v5.8.0+0
  [8e850ede] nghttp2_jll v1.48.0+0
  [3f19e933] p7zip_jll v17.4.0+0
Info Packages marked with ⌅ have new versions available but compatibility constraints restrict them from upgrading. To see why use `status --outdated -m`

[odow]

sorry for my not having time to revisit this until today

No need. You're volunteering your time :smile:

The crash looks like it happens at the same time as https://github.com/ralna/spral/issues/81.

You could try setting set_attribute(model, "spral_print_level", 5) (I don't know if 5 is a good value).

Otherwise, just use WSL or a different linear solver.

[amontoison]

@jfowkes I added SPRAL as a linear solver of Ipopt when I recompiled it with Yggdrasil. Because it's easy to use now, we have some Windows users and Spral is only working in WSL.

[jfowkes]

This indeed looks to be the same issue as ralna/spral#81, what I don't understand is why we're only seeing it on Windows and not elsewhere?

[jfowkes]

We have tested this on SPRAL on Windows outside of IPOPT and are unable to reproduce this issue.

[amontoison]

What version of GCC did you use? SPRAL is compiled with GNU compilers 9.1 and Ipopt is compiled with the oldest version (8.1) that use libgfortran5.

[chriscoey]

The linked issue https://github.com/ralna/spral/issues/81 has been fixed. So perhaps this issue is also fixed now?

[amontoison]

I added a test here and it works on all platforms.

[odow]

Closing because https://github.com/jump-dev/Ipopt.jl/pull/380 is passing tests on Windows.

I haven't tagged a release yet, but please re-open if you have a reproducible example on the latest master.

Edit: this shouldn't need a new release. It should work with Ipopt.jl v1.4.2