Closed odow closed 2 months ago
Exciting: I'll try to reproduce it
Hello, just +1ing myself here as I might be hitting the very same issue.
Running HiGHS 1.7.2 (git hash: 5ce7a2753): Copyright (c) 2024 HiGHS under MIT licence terms
Coefficient ranges:
Matrix [1e-05, 2e+05]
Cost [1e+00, 1e+00]
Bound [0e+00, 0e+00]
RHS [1e-01, 6e+03]
Presolving model
85619 rows, 142837 cols, 551793 nonzeros 0s
[...nothing happens...]
Switching presolve
off solves the issue (although for this model size the algorithms get annoyingly slow so I didn't get an applicable result yet).
Also I unfortunately cannot share the model. I'd try to produce one untainted with sensitive data in case it would help reproducing the issue here, but it's gonna need some nontrivial time investment -- please let me know in case that would be needed.
Hello, just +1ing myself here as I might be hitting the very same issue.
Thanks. I'll see what I get from @odow's example, and there's another user who may have the same issue
I've reproduced the behaviour in release
, and it triggers an assert in debug
so there's a start...
Presolve is trying to fix a variable to -inf
. Presumably related to the LP being unbounded
Closed by #1885
@jajhall Hello, is there please any estimate on if/when this is going to get released? (I'd love to enjoy the fix in Julia via HiGHS.jl :) )
Within a month, I'd expect
OK perfect, thanks! ( I'll try to build this locally then, unfortunately this issue was a bit of a blocker so certainly worth trying :) )
Every time or once in a while the github actions could extract and make available the resulting binaries.
Every time or once in a while the github actions could extract and make available the resulting binaries.
Yes, but they wouldn't be fully tested, or correspond to an identifiable release.
Clearly anyone can build latest
from source, but that's up to them!
@jajhall is there please any plan re the release?
(Apologies for likely unnecessary spam here, but not having this fix released has become a source of minor annoyances (telling users to "just compile HiGHS with BinaryBuilder.jl custom script or use Gurobi" ain't very good). So just kinda wanted to ask for status. Thanks for any info! )
We've got quite a lot of Python changes that we'll need to release as the ultimate test. Hopefully before 7 October
ok great! thanks for the info!
@jajhall Hello, just reporting here that the issue is not solved on my side. The model where I can reproduce this is unfortunately huge and slightly non-publishable, is there any good way to export a JuMP/HiGHS model so that you could have a look at it?
Running HiGHS 1.7.2 (git hash: f7c79b3b3): Copyright (c) 2024 HiGHS under MIT licence terms
Coefficient ranges:
Matrix [1e-05, 2e+05]
Cost [1e+00, 1e+00]
Bound [0e+00, 0e+00]
RHS [4e-04, 8e+03]
Presolving model
85619 rows, 142838 cols, 551809 nonzeros 0s
[...never finishes]
(the reported f7c79b3b3 is the latest commit on latest
)
Backtrace from the stuck solver:
#0 0x00007f27502339cb in presolve::HPresolve::updateColImpliedBounds(int, int, double) () from /home/exa/.julia/dev/HiGHS_jll/override/lib/libhighs.so
No symbol table info available.
#1 0x00007f2750234264 in presolve::HPresolve::recomputeColImpliedBounds(int) () from /home/exa/.julia/dev/HiGHS_jll/override/lib/libhighs.so
No symbol table info available.
#2 0x00007f2750238f7c in presolve::HPresolve::substitute(int, int, double) () from /home/exa/.julia/dev/HiGHS_jll/override/lib/libhighs.so
No symbol table info available.
#3 0x00007f27502c8188 in presolve::HPresolve::aggregator(presolve::HighsPostsolveStack&) () from /home/exa/.julia/dev/HiGHS_jll/override/lib/libhighs.so
No symbol table info available.
#4 0x00007f275023e667 in presolve::HPresolve::presolve(presolve::HighsPostsolveStack&) () from /home/exa/.julia/dev/HiGHS_jll/override/lib/libhighs.so
No symbol table info available.
#5 0x00007f27502c5855 in presolve::HPresolve::run(presolve::HighsPostsolveStack&) () from /home/exa/.julia/dev/HiGHS_jll/override/lib/libhighs.so
No symbol table info available.
#6 0x00007f27501cfabf in PresolveComponent::run() () from /home/exa/.julia/dev/HiGHS_jll/override/lib/libhighs.so
No symbol table info available.
#7 0x00007f27501d6a89 in Highs::runPresolve(bool, bool) () from /home/exa/.julia/dev/HiGHS_jll/override/lib/libhighs.so
No symbol table info available.
#8 0x00007f27501de4a4 in Highs::run() () from /home/exa/.julia/dev/HiGHS_jll/override/lib/libhighs.so
No symbol table info available.
#9 0x00007f274f83c405 in Highs_run () at /home/exa/.julia/packages/HiGHS/MQAyL/src/gen/libhighs.jl:311
No locals.
#10 julia_optimize!_8615 () at /home/exa/.julia/packages/HiGHS/MQAyL/src/MOI_wrapper.jl:1915
No locals.
#11 0x00007f274f83c701 in optimize! () at /home/exa/.julia/packages/MathOptInterface/jqDoD/src/Bridges/bridge_optimizer.jl:367
No locals.
#12 optimize! () at /home/exa/.julia/packages/MathOptInterface/jqDoD/src/MathOptInterface.jl:122
No locals.
[..., continues to Julia]
I'll try to make a build where local variables will be visible.
Managed to get a fuller backtrace, I can produce debug backgraces quite easily now. Let me know in case you want to have a watch/breakpoint somewhere or so. Current view on the stuck presolve is:
#0 HighsLinearSumBounds::updatedImplVarUpper (this=this@entry=0x7f16de1fe648, sum=15560,
var=75903, coefficient=-1, oldImplVarUpper=oldImplVarUpper@entry=inf,
oldImplVarUpperSource=oldImplVarUpperSource@entry=-1)
at /workspace/srcdir/HiGHS/src/util/HighsLinearSumBounds.cpp:245
oldVUpper = <optimized out>
vUpper = <optimized out>
#1 0x00007f17625d6a5b in _ZN8presolve9HPresolve18changeImplColUpperEidi (
this=this@entry=0x7f16de1fe390, col=<optimized out>, col@entry=75903,
newUpper=<optimized out>, originRow=originRow@entry=422927)
at /workspace/srcdir/HiGHS/src/presolve/HPresolve.cpp:1855
nonzero = <optimized out>
__for_range = @0x7f16de1fdf30: {<No data fields>}
__for_begin = <optimized out>
__for_end = <optimized out>
oldImplUpper = inf
oldUpperSource = -1
newImpliedFree = false
#2 0x00007f17625db839 in _ZN8presolve9HPresolve22updateColImpliedBoundsEiid (
this=this@entry=0x7f16de1fe390, row=422927, col=75903, val=<optimized out>)
at /workspace/srcdir/HiGHS/src/presolve/HPresolve.cpp:618
impliedBound = <optimized out>
residualMinAct = <optimized out>
rowUpper = <optimized out>
rowLower = <optimized out>
#3 0x00007f17625dbd04 in _ZN8presolve9HPresolve25recomputeColImpliedBoundsEi (
this=this@entry=0x7f16de1fe390, row=row@entry=422927)
at /workspace/srcdir/HiGHS/src/presolve/HPresolve.cpp:726
nonz = <optimized out>
__for_range = @0x7f16de1fe060: {<No data fields>}
__for_begin = <optimized out>
__for_end = <optimized out>
it = {_M_node = 0x51534cf0}
affectedCols = {_M_t = {_M_impl = {_M_key_compare = {<No data fields>}, _M_header = {
_M_color = _S_red, _M_parent = 0x53c3fae0, _M_left = 0x5177e6e0,
_M_right = 0x53e65090}, _M_node_count = 70237}}}
#4 0x00007f17625e09bc in _ZN8presolve9HPresolve10substituteEiid (
this=this@entry=0x7f16de1fe390, row=row@entry=151190, col=col@entry=165457, rhs=0)
at /workspace/srcdir/HiGHS/src/presolve/HPresolve.cpp:2467
colrow = 422927
colval = <optimized out>
colpos = <optimized out>
scale = -1
coliter = 847611
pos = <optimized out>
substrowscale = -1
#5 0x00007f176269e937 in _ZN8presolve9HPresolve10aggregatorERNS_19HighsPostsolveStackE (
this=this@entry=0x7f16de1fe390, postsolve_stack=...)
at /workspace/srcdir/HiGHS/src/presolve/HPresolve.cpp:4924
rhs = 0
rowType = kEq
fillin = <optimized out>
rhs = 0
row = 151190
col = 165457
nzPos = <optimized out>
maxVal = <optimized out>
rowType = kEq
i = 899
logging_on = false
nfail = 0
#6 0x00007f17625e6087 in _ZN8presolve9HPresolve8presolveERNS_19HighsPostsolveStackE (
this=this@entry=0x7f16de1fe390, postsolve_stack=...)
at /workspace/srcdir/HiGHS/src/presolve/HPresolve.cpp:4323
__result = <optimized out>
currSize = <optimized out>
report = {__this = 0x7f16de1fe390}
numParallelRowColCalls = 0
tryProbing = false
domcolAfterProbingCalled = false
trySparsify = false
numCliquesBeforeProbing = -1
dependentEquationsCalled = false
lastPrintSize = 228457
#7 0x00007f176269ef25 in _ZN8presolve9HPresolve3runERNS_19HighsPostsolveStackE (
this=this@entry=0x7f16de1fe390, postsolve_stack=...)
at /workspace/srcdir/HiGHS/src/presolve/HPresolve.cpp:4535
reportReductions = <optimized out>
#8 0x00007f17625a814f in _ZN17PresolveComponent3runEv (this=this@entry=0x53747918)
at /workspace/srcdir/HiGHS/src/presolve/PresolveComponent.cpp:42
presolve = {model = 0x53747960, options = 0x537438e8, timer = 0x53743758,
mipsolver = 0x0, primal_feastol = 9.9999999999999995e-08, run_clock = 0,
Avalue = {<No data fields>}, Arow = {<No data fields>}, Acol = {<No data fields>},
colhead = {<No data fields>}, Anext = {<No data fields>},
Aprev = {<No data fields>}, rowroot = {<No data fields>},
ARleft = {<No data fields>}, ARright = {<No data fields>},
rowsize = {<No data fields>}, rowsizeInteger = {<No data fields>},
rowsizeImplInt = {<No data fields>}, colsize = {<No data fields>},
rowpositions = {<No data fields>}, freeslots = {<No data fields>},
implColLower = {<No data fields>}, implColUpper = {<No data fields>},
colLowerSource = {<No data fields>}, colUpperSource = {<No data fields>},
rowDualLower = {<No data fields>}, rowDualUpper = {<No data fields>},
implRowDualLower = {<No data fields>}, implRowDualUpper = {<No data fields>},
rowDualLowerSource = {<No data fields>}, rowDualUpperSource = {<No data fields>},
colImplSourceByRow = {<No data fields>},
implRowDualSourceByCol = {<No data fields>}, impliedRowBounds = {
sumLowerOrig = {<No data fields>}, sumUpperOrig = {<No data fields>},
numInfSumLowerOrig = {<No data fields>},
numInfSumUpperOrig = {<No data fields>}, sumLower = {<No data fields>},
sumUpper = {<No data fields>}, numInfSumLower = {<No data fields>},
numInfSumUpper = {<No data fields>}, varLower = 0x48c41a80,
varUpper = 0x4a9900c0, implVarLower = 0x55f6e4c0, implVarUpper = 0x2e515990,
implVarLowerSource = 0x2c1863a0, implVarUpperSource = 0x261f4030},
impliedDualRowBounds = {sumLowerOrig = {<No data fields>},
sumUpperOrig = {<No data fields>}, numInfSumLowerOrig = {<No data fields>},
numInfSumUpperOrig = {<No data fields>}, sumLower = {<No data fields>},
sumUpper = {<No data fields>}, numInfSumLower = {<No data fields>},
numInfSumUpper = {<No data fields>}, varLower = 0x30738bc0,
varUpper = 0x30a73090, implVarLower = 0x30dad560, implVarUpper = 0x43f48fd0,
implVarLowerSource = 0x28bb2890, implVarUpperSource = 0x34a5da60},
changedRowIndices = {<No data fields>}, changedRowFlag = {<No data fields>},
changedColIndices = {<No data fields>}, changedColFlag = {<No data fields>},
substitutionOpportunities = {<No data fields>}, equations = {_M_t = {_M_impl = {
_M_key_compare = {<No data fields>}, _M_header = {_M_color = _S_red,
_M_parent = 0x528512d0, _M_left = 0x258c5590, _M_right = 0x299e93f0},
_M_node_count = 41958}}}, eqiters = {<No data fields>},
shrinkProblemEnabled = true, reductionLimit = 18446744073709551615,
singletonRows = {<No data fields>}, singletonColumns = {<No data fields>},
rowDeleted = {<No data fields>}, colDeleted = {<No data fields>},
numProbes = {<No data fields>}, probingContingent = 723507328,
probingNumDelCol = -783769463, numProbed = 32535, numDeletedRows = 381106,
numDeletedCols = 171778, oldNumCol = 85142, oldNumRow = 42410,
probingEarlyAbort = false, presolve_status_ = kNotSet, analysis_ = {
model = 0x53747960, options = 0x537438e8, allow_rule = 0x0,
numDeletedRows = 0x7f16de1fe980, numDeletedCols = 0x7f16de1fe984,
original_num_col_ = 256469, original_num_row_ = 423065,
allow_rule_ = {<No data fields>}, allow_logging_ = false, logging_on_ = false,
log_rule_type_ = -1, num_deleted_rows0_ = 0, num_deleted_cols0_ = 0,
presolve_log_ = {rule = {<No data fields>}}}}
#9 0x00007f1762571df5 in _ZN5Highs11runPresolveEbb (this=this@entry=0x53742e50,
force_lp_presolve=force_lp_presolve@entry=true,
force_presolve=force_presolve@entry=false)
at /workspace/srcdir/HiGHS/src/lp_data/Highs.cpp:3340
original_lp = @0x53742f88: {num_col_ = 256469, num_row_ = 423065,
col_cost_ = {<No data fields>}, col_lower_ = {<No data fields>},
col_upper_ = {<No data fields>}, row_lower_ = {<No data fields>},
row_upper_ = {<No data fields>}, a_matrix_ = {format_ = kColwise,
num_col_ = 256469, num_row_ = 423065, start_ = {<No data fields>},
p_end_ = {<No data fields>}, index_ = {<No data fields>},
value_ = {<No data fields>}}, sense_ = kMaximize, offset_ = 0,
model_name_ = <incomplete type>, objective_name_ = <incomplete type>,
new_col_name_ix_ = 0, new_row_name_ix_ = 0, col_names_ = {<No data fields>},
row_names_ = {<No data fields>}, integrality_ = {<No data fields>}, col_hash_ = {
name2index = {_M_h = {_M_buckets = 0x53743148, _M_bucket_count = 1,
_M_before_begin = {_M_nxt = 0x0}, _M_element_count = 0, _M_rehash_policy = {
_M_max_load_factor = 1, _M_next_resize = 0}, _M_single_bucket = 0x0}}},
row_hash_ = {name2index = {_M_h = {_M_buckets = 0x53743180, _M_bucket_count = 1,
_M_before_begin = {_M_nxt = 0x0}, _M_element_count = 0, _M_rehash_policy = {
_M_max_load_factor = 1, _M_next_resize = 0}, _M_single_bucket = 0x0}}},
user_bound_scale_ = 0, user_cost_scale_ = 0, scale_ = {strategy = 0,
has_scaling = false, num_col = 0, num_row = 0, cost = 0,
col = {<No data fields>}, row = {<No data fields>}}, is_scaled_ = false,
is_moved_ = false, cost_row_location_ = -1, has_infinite_cost_ = false, mods_ = {
save_non_semi_variable_index = {<No data fields>},
save_inconsistent_semi_variable_index = {<No data fields>},
save_inconsistent_semi_variable_lower_bound_value = {<No data fields>},
save_inconsistent_semi_variable_upper_bound_value = {<No data fields>},
save_inconsistent_semi_variable_type = {<No data fields>},
save_relaxed_semi_variable_lower_bound_index = {<No data fields>},
save_relaxed_semi_variable_lower_bound_value = {<No data fields>},
save_tightened_semi_variable_upper_bound_index = {<No data fields>},
save_tightened_semi_variable_upper_bound_value = {<No data fields>},
save_inf_cost_variable_index = {<No data fields>},
save_inf_cost_variable_cost = {<No data fields>},
save_inf_cost_variable_lower = {<No data fields>},
save_inf_cost_variable_upper = {<No data fields>}}}
start_presolve = 0.012864112854003906
presolve_return_status = kNotPresolved
#10 0x00007f1762579b5c in _ZN5Highs3runEv (this=0x53742e50)
at /workspace/srcdir/HiGHS/src/lp_data/Highs.cpp:1238
force_lp_presolve = true
to_presolve_time = <optimized out>
factor_pivot_threshold = <optimized out>
have_optimal_solution = <optimized out>
lp_presolve_requires_basis_postsolve = true
from_presolve_time = <optimized out>
min_highs_debug_level = 0
possibly_use_log_dev_level_2 = false
log_dev_level = 0
output_flag = true
use_log_dev_level = 0
use_output_flag = true
check_debug_run_call_num = -103757
check_num_col = -317
check_num_row = -714
undo_mods = false
return_status = kOk
call_status = kOk
use_simplex_or_ipm = <optimized out>
incumbent_lp = @0x53742f88: {num_col_ = 256469, num_row_ = 423065,
col_cost_ = {<No data fields>}, col_lower_ = {<No data fields>},
col_upper_ = {<No data fields>}, row_lower_ = {<No data fields>},
row_upper_ = {<No data fields>}, a_matrix_ = {format_ = kColwise,
num_col_ = 256469, num_row_ = 423065, start_ = {<No data fields>},
p_end_ = {<No data fields>}, index_ = {<No data fields>},
value_ = {<No data fields>}}, sense_ = kMaximize, offset_ = 0,
model_name_ = <incomplete type>, objective_name_ = <incomplete type>,
new_col_name_ix_ = 0, new_row_name_ix_ = 0, col_names_ = {<No data fields>},
row_names_ = {<No data fields>}, integrality_ = {<No data fields>}, col_hash_ = {
name2index = {_M_h = {_M_buckets = 0x53743148, _M_bucket_count = 1,
_M_before_begin = {_M_nxt = 0x0}, _M_element_count = 0, _M_rehash_policy = {
_M_max_load_factor = 1, _M_next_resize = 0}, _M_single_bucket = 0x0}}},
row_hash_ = {name2index = {_M_h = {_M_buckets = 0x53743180, _M_bucket_count = 1,
_M_before_begin = {_M_nxt = 0x0}, _M_element_count = 0, _M_rehash_policy = {
_M_max_load_factor = 1, _M_next_resize = 0}, _M_single_bucket = 0x0}}},
user_bound_scale_ = 0, user_cost_scale_ = 0, scale_ = {strategy = 0,
has_scaling = false, num_col = 0, num_row = 0, cost = 0,
col = {<No data fields>}, row = {<No data fields>}}, is_scaled_ = false,
is_moved_ = false, cost_row_location_ = -1, has_infinite_cost_ = false, mods_ = {
save_non_semi_variable_index = {<No data fields>},
save_inconsistent_semi_variable_index = {<No data fields>},
save_inconsistent_semi_variable_lower_bound_value = {<No data fields>},
save_inconsistent_semi_variable_upper_bound_value = {<No data fields>},
save_inconsistent_semi_variable_type = {<No data fields>},
save_relaxed_semi_variable_lower_bound_index = {<No data fields>},
save_relaxed_semi_variable_lower_bound_value = {<No data fields>},
save_tightened_semi_variable_upper_bound_index = {<No data fields>},
save_tightened_semi_variable_upper_bound_value = {<No data fields>},
save_inf_cost_variable_index = {<No data fields>},
save_inf_cost_variable_cost = {<No data fields>},
save_inf_cost_variable_lower = {<No data fields>},
save_inf_cost_variable_upper = {<No data fields>}}}
log_options = @0x53743c48: {log_stream = 0x0, output_flag = 0x53743ab0,
log_to_console = 0x53743ab1, log_dev_level = 0x53743adc, user_log_callback = 0x0,
user_log_callback_data = 0x0, user_callback = {_M_invoker = 0x53743ca0},
user_callback_data = 0x0, user_callback_active = false}
no_incumbent_lp_solution_or_basis = false
initial_time = 0.01286005973815918
this_presolve_time = <optimized out>
this_solve_presolved_lp_time = -1
this_postsolve_time = -1
this_solve_original_lp_time = -1
postsolve_iteration_count = -1
ipx_no_crossover = false
solveLp = {__this = 0x53742e50, __use_log_dev_level = @0x7f16de1fed58,
__use_output_flag = @0x7f16de1fed52, __call_status = @0x7f16de1fed54,
__log_dev_level = @0x7f16de1fed5c, __output_flag = @0x7f16de1fed53}
unconstrained_lp = <optimized out>
lp_solve_final_time = <optimized out>
this_solve_time = <optimized out>
#11 0x00007f1761c3c405 in Highs_run ()
at /home/exa/.julia/packages/HiGHS/MQAyL/src/gen/libhighs.jl:311
No locals.
#12 julia_optimize!_8615 () at /home/exa/.julia/packages/HiGHS/MQAyL/src/MOI_wrapper.jl:1915
No locals.
#13 0x00007f1761c3c701 in optimize! ()
at /home/exa/.julia/packages/MathOptInterface/jqDoD/src/Bridges/bridge_optimizer.jl:367
No locals.
#14 optimize! ()
at /home/exa/.julia/packages/MathOptInterface/jqDoD/src/MathOptInterface.jl:122
No locals.
#15 julia_optimize!_10419 ()
at /home/exa/.julia/packages/MathOptInterface/jqDoD/src/Utilities/cachingoptimizer.jl:321
No locals.
#16 0x00007f1761c70e67 in jfptr_optimizeNOT._10420 ()
from /home/exa/.julia/compiled/v1.10/HiGHS/WFic5_r1cQ1.so
No symbol table info available.
#17 0x00007f17d0446a0e in _jl_invoke (world=<optimized out>,
mfunc=0x7f1761dfee50 <jl_system_image_data+1582160>, nargs=1, args=0x7f16de1ff570,
F=0x7f17688f68c0 <jl_system_image_data+15191072>)
at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/gf.c:2894
[....]
I'm attaching a .mps of the system just before the presolve failed, hopefully it helps debugging: big_model.mps.gz
I was unfortunately unable to reduce the issue to anything smaller.
(btw this might deserve another issue, please ping me if I should open one)
Here's an MPS that does not terminate when presolve is active: bug.mps.txt
Found by @Rose-max111 in https://github.com/jump-dev/HiGHS.jl/issues/221
With
presolve=off
I get:With presolve on, it stalls at: