Request: Documentation of solution status

[karldw]

I've gotten a status message of Unwelcome IPX status of Unknown: basis is not valid; solution is valid; run_crossover is "off" I'm extremely far from being an optimization expert -- I'm unclear if this message represents a failure or not. HiGHS.jl converts it to OTHER_ERROR. Would it be possible to provide more documentation of whether this is an issue?

Context

I'm trying to use HiGHS to run the models from Xu et al. (2024), which originally used Gurobi. My hope is to use HiGHS and compare with that paper's replication files.

Here's my HiGHS log for the scenario CA_ncp_2030_currentpolicy_10ci_nocip, which is inside the CA_Inputs_and_Results.zip file.

HiGHS log

``` Running HiGHS 1.7.0 (git hash: 50670fd4c): Copyright (c) 2024 HiGHS under MIT licence terms Coefficient ranges: Matrix [1e-03, 1e+03] Cost [2e-01, 3e+08] Bound [0e+00, 0e+00] RHS [2e-01, 3e+08] Presolving model 2837537 rows, 1954724 cols, 10478628 nonzeros 2s 2194841 rows, 1502561 cols, 9500379 nonzeros 4s 2144550 rows, 1470426 cols, 9625691 nonzeros 10s Presolve : Reductions: rows 2144550(-1995681); columns 1470426(-1671887); elements 9625691(-4859331) Solving the presolved LP IPX model has 2144550 rows, 1488570 columns and 9643835 nonzeros Input Number of variables: 1488570 Number of free variables: 199486 Number of constraints: 2144550 Number of equality constraints: 360025 Number of matrix entries: 9643835 Matrix range: [7e-04, 1e+03] RHS range: [1e+00, 2e+08] Objective range: [2e-01, 3e+06] Bounds range: [5e-13, 2e+05] Preprocessing Dualized model: no Number of dense columns: 208 Range of scaling factors: [6.25e-02, 8.00e+00] IPX version 1.0 Interior Point Solve Iter P.res D.res P.obj D.obj mu Time 0 5.69e+04 1.17e+05 2.95836617e+11 -1.14914602e+14 6.66e+09 14s Constructing starting basis... LU pivot tolerance tightened to 0.3 LU pivot tolerance tightened to 0.9 1 5.67e+04 1.16e+05 1.70750653e+11 -1.15270011e+14 6.66e+09 753s <...> 79 1.96e-06 1.56e-03 1.67715856e+10 6.92604238e+09 2.94e+03 22691s Summary Runtime: 22691.14s Status interior point solve: no progress Status crossover: not run WARNING: Ipx: Stopped WARNING: Ipx: IPM no progress WARNING: No progress: primal objective value = 1.677e+10 WARNING: No progress: max absolute primal residual = 1.261e-05 WARNING: No progress: max absolute dual residual = 0.001563 WARNING: Unwelcome IPX status of Unknown: basis is not valid; solution is valid; run_crossover is "off" Model status : Unknown IPM iterations: 79 Objective value : 2.2198320287e+10 HiGHS run time : 22702.91 LP solved for primal ```

Comparing a diff of the results with the original results in the replication files, everything seems identical (to many decimal places) except the status.csv file. (Though this comment notes that "Gurobi excludes constants from solver reported objective function value".)

git diff Results/status.csv:

 Status,Solve,Objval
-LOCALLY_SOLVED,1589.5961499214172,1.7612290239939682e10
+OTHER_ERROR,22759.382441043854,2.2198320287495457e10

Versions and settings

Versions: - HiGHS: 1.7.0 - HiGHS.jl: 1.9.0 - Julia: 1.6.7 ```yaml # HiGHS Solver Parameters # Common solver settings Feasib_Tol: 1.0e-05 # Primal feasibility tolerance # [type: double, advanced: false, range: [1e-10, inf], default: 1e-07] Optimal_Tol: 1.0e-05 # Dual feasibility tolerance # [type: double, advanced: false, range: [1e-10, inf], default: 1e-07] TimeLimit: 1.0e23 # Time limit # [type: double, advanced: false, range: [0, inf], default: inf] Pre_Solve: "choose" # Presolve option: "off", "choose" or "on" # [type: string, advanced: false, default: "choose"] Method: "ipm" # HiGHS-specific solver settings # Solver option: "simplex", "choose" or "ipm" # [type: string, advanced: false, default: "choose"] # run the crossover routine for ipx # [type: string, advanced: "on", range: {"off", "on"}, default: "off"] run_crossover: "off" ```

Let me also say thank you for all of your work -- it's immensely useful to have an open source solver like HiGHS!

[jajhall]

The HIGHS IPM solver has simply failed. The following suggest that the problem is numerically challenging

LU pivot tolerance tightened to 0.3 LU pivot tolerance tightened to 0.9

The final values of 1.67715856e+10 and 6.92604238e+09 for the primal and dual objective values have a big gap, and whilst

max absolute primal residual = 1.261e-05

isn't so bad

max absolute dual residual = 0.001563

is less encouraging.

This isn't a bug, just a failure

Thanks for your kind words, we're used to people only getting in touch when things go wrong!

[karldw]

This is enormously helpful. Thank you!